A typographical error would mishandle the case where a caller explicitly
tries to remove a child that is not actually part of the node. This case
is never invoked in practice, and so the bug would will never trigger.
Commit bf013c02ad added support
for incorporating a commit ID into the compiled version string
but did so in a way that did not follow the semantic versioning
standard.
This commit corrects that flaw by moving the commit ID into the
"metadata" part of the version string and properly handles the
case where the commit hash cannot be retrieved.
The pathfinding engine built into the code has several configurable
parameters to adjust the depth of the paths indexed and explored.
These parameters can dramatically impact the performance and memory
consumption of a server; higher values can result in resource usage
increasing exponentially.
These default values were decided early and somewhat arbitrarily at
a time when the network and the size of the network state were much
smaller.
This commit adjusts the default values to reduce the depth of paths
to more reasonable levels; unless explicitly overriden, the changes
mean that pathfinding operations will return fewer, shallower paths
than previous versions of the software.
This commit corrects a technical flaw that was introduced with commit
7c12f01358: as written, a mutex that is
intended to help provide synchronization for multiple threads as they
are each walking the map, is declared so that each thread is passed a
dangling reference to a unique mutex.
This commit hoists the mutex outside the thread creation loop, so all
threads use a single mutex and eliminating the dangling reference.
The nodestore includes a built-in cache to reduce the disk I/O
load but, by default, this cache was not initialized unless it
was explicitly configured by the server operator.
This commit introduces sensible defaults based on the server's
configured node size.
It remains possible to completely disable the cache if desired
by explicitly configuring it the cache size and age parameters
to 0:
[node_db]
...
cache_size = 0
cache_age = 0
- Only duplicate records from archive to writable during online_delete.
- Log duration of nodestore reads.
- Include nodestore counters in perf_log output.
- Remove gratuitous nodestore activity counting.
- Report initial sync duration in server_info and perfLog.
- Report state_accounting in perfLog.
- Make state_accounting durations more accurate.
- Parallel ledger loader.
- Config parameter to load ledgers on start.
1) Don't acquire so many nodes per pass. It's likely
far more than we need.
2) Right-size the finishedReads_ vector on passes other
than just the first.
* Sort by fee level (which is the current behavior) then by transaction
ID (hash).
* Edge case when the account at the end of the queue submits a higher
paying transaction to walk backwards and compare against the cheapest
transaction from a different account.
* Use std::if_any to simplify the JobQueue::isOverloaded loop.
* Log load fee values (at debug) received from validations.
* Log remote and cluster fee values (at trace) when changed.
* Refactor JobQueue::isOverloaded to return sooner if overloaded.
* Refactor Transactor::checkFee to only compute fee if ledger is open.
This flag, if present, suppresses the output of incoming
trustlines in default state.
This is primarily motivated by observing that users of Xumm often
have many unwanted incoming trustlines in a default state, which are
not useful in the vast majority of cases.
Being able to suppress those when doing `account_lines` saves bandwidth
and resources.
This commit implements partitioned unordered maps and makes it possible
to traverse such a map in parallel, allowing for more efficient use of
CPU resources.
The `CachedSLEs`, `TaggedCache`, and `KeyCache` classes make use of the
new functionality, which should improve performance.
The pathfinding engine requires pre-building large tables which is a
resource-intensive operation. Typically, one would not expect that a
server configured as a validator would also support pathfinding APIs
and so, building those tables by default wastes resources.
This commit, if merged, will disable pathfinding on servers that are
configured as validators, unless the server operator opts to support
it explicitly, by configuring the `[path_search_max]` parameter.
Validator operators that wish to support pathfinding on a validator
and want to use the default values can add the following stanza to
their server's configuration file:
[path_search_max]
7
The priority of different types of jobs was set back in the early
days of development, based on insight and observations that don't
necessarily apply any longer.
Specifically, job types used by the server to sync to the network
were being treated as lower priority than client requests, making
it more difficult to regain sync.
This commit adjusts the priority of several jobs and should allow
servers to prioritize resynchronizing to the network over serving
clients.
The existing calculation would limit the maximum number of threads
that would be created by default to at most 6; this may have been
reasonable a few years ago, but given both the load on the network
as of today and the increase in the number of CPU cores, the value
should be revisited.
This commit, if merged, changes the default calculation for nodes
that are configured as `large` or `huge` to allow for up to twelve
threads.
The "sweep interval" is the amount of time between successive sweeps of
of various in-memory data structures to remove stale items.
Prior to this commit, the interval was automatically adjusted, based on
the value of the `[node_size]` option in a server's configuration file.
If merged, this commit introduces a new configuration option that makes
it possible for a server operator to adjust the sweep interval and make
a CPU/memory tradeoff:
[sweep_interval]
<integer>
The specified value represents the number of seconds between successive
sweeps. The range of valid values is between 10 and 600.
Important operator notes:
This is an advanced configuration option that should not be used unless
there is empirical data which suggests that the default sweep frequency
is either resulting in performance problems or is causing undue load to
the server.
Note that adjusting the sweep interval may not have the intended effect
on the server. Lower values will not always translate to a reduction of
memory usage and higher values will not always translate to a reduction
of CPU usage and/or load.
The performance characteristics of `std::unordered_map` are better
than `std::map` and the former should be preferred when the strict
ordering of the latter is not required.
* Only require adding the new feature names in one place. (Also need to
increment a counter, but a check on startup will catch that.)
* Allows rippled to have the code to support a given amendment, but
not vote for it by default. This allows the amendment to be enabled in
a future version without necessarily amendment blocking these older
versions.
* The default vote is carried with the amendment name in the list of
supported amendments.
* The amendment table is constructed with the amendment and default
vote.
* Also clean up some formatting in the Windows instructions
* Changed the recommended version for Windows to 1.1.1L after deeper
checking uncovered some build issues.
* Patch the soci unsigned-types.h file. If no changes are made, delete
the patched file and exit. If there are changes, backup the original
and replace it with the patched file.
* Fixes#3885
Patch Rocksdb only once:
* The repeated patches do not appear to affect build times, but avoiding
unnecessary copies is good for its own sake.
There are two mutexes in ValidatorSite: `site_mutex_` and `state_mutex_`. Some
function end up locking both mutexes. However, depending on the call, the
mutexes could be locked in different orders, resulting in deadlocks.
If both mutexes are locked, this patch always locks the `sites_mutex_` first.
The existing logic involves every server sending every transaction
that it receives to all its peers (except the one that it received
a transaction from).
This commit instead uses a randomized algorithm, where a node will
randomly select peers to relay a given transaction to, caching the
list of transaction hashes that are not relayed and forwading them
to peers once every second. Peers can then determine whether there
are transactions that they have not seen and can request them from
the node which has them.
It is expected that this feature will further reduce the bandwidth
needed to operate a server.
The existing license file contains copyright statements and license
snippets referencing code that has long since been removed from the
codebase and which are not necessary any longer.
The general copyright statement for the ISC License is also changed
from "Ripple Labs" which is one contributor to the codebase, to the
more general "XRP Ledger Developers".
Remaining code which was originally taken from Bitcoin includes the
relevant copyright statement(s) inline.
To aid in the automated detection of the license type by GitHub and
tools like `licensee`, the following statement which referenced the
ISC license, and which was listed at the bottom of the file, is now
incorporated at the top:
>The accompanying files incorporate work covered by the following copyright
>and previous license notice:
>
>Copyright (c) 2011 Arthur Britto, David Schwartz, Jed McCaleb,
>Vinnie Falco, Bob Way, Eric Lombrozo, Nikolaos D. Bougalis, Howard Hinnant
This commit removes the `ltINVALID` pseudo-type identifier from
`LedgerEntryType` and the `ttINVALID` pseudo-type identifier from
`TxType` and includes several small additional improvements that
help to simplify the code base.
It also improves the documentation `LedgerEntryType` and `TxType`,
which was all over the place, and highlights some important caveats
associated with making changes to the ledger and transaction type
identifiers.
The commit also adds a safety check to the `KnownFormats<>` class,
that will catch the the accidental reuse of format identifiers.
Ideally, this should be done at compile time but C++ does not (yet?)
allow for the sort of introspection that would enable this.
The legacy functions `cdirFirst` and `dirFirst` were mostly
identical; the differences were only type-related. The same
situation existed with `cdirNext` and `dirNext`.
This commit removes the duplicated code by introducing new
template functions that abstract away the differences that
are present between each pair of functions.
This commit also improves the naming of function arguments,
helping to elucidate their purpose & use and to make the
code self-documenting.
The Negative UNL is a feature of the XRP Ledger consensus protocol that
improves liveness (the network's ability to make forward progress) during
a partial outage. Using the Negative UNL, servers adjust their effective
UNLs based on which validators are currently online and operational, so
that a new ledger version can be declared validated even if several trusted
validators are offline.
The Negative UNL has no impact on how the network processes transactions
or what transactions' outcomes are, except that it improves the network's
ability to declare outcomes final during some types of partial outages.
The feature was originally introduced with version **1.6.0** but it was
only possible to manually enable this. If merged, this commit introduces
the amendment associated with the feature so that server operators can
vote on whether to enable this feature.
For more details, please see https://xrpl.org/negative-unl.html
This commit closes#3898.
Under some circumstances, it is possible to induce an out-of-bounds
memory read in the base58 decoder.
This commit addresses this issue.
Acknowledgements:
Guido Vranken for discovering and responsibly disclosing this issue.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to
responsibly disclose any issues they may find.
Ripple is generously sponsoring a bug bounty program for the
rippled project. For more information please visit:
https://ripple.com/bug-bounty
The HardenedValidations amendment introduces additional fields
in validations:
- `sfValidatedHash`, if present, is the hash the of last ledger that
the validator considers to be fully validated.
- `sfCookie`, if present, is a 64-bit cookie (the default
implementation selects it randomly at startup but other
implementations are possible), which can be used to improve the
detection and classification of duplicate validations.
- `sfServerVersion`, if present, reports the version of the software
that the validator is running. By surfacing this information,
server operators gain additional insight about variety of software
on the network.
If merged, this commit fixes#3797 by adding the fields to the
`validations` stream as shown below:
- `sfValidateHash` as `validated_hash`: a 256-bit hex string;
- `sfCookie` as `cookie`: a 64-bit integer as a string; and
- `sfServerVersion` as `server_version`: a 64-bit integer as
a string.
With this amendment, the CheckCash transaction creates a TrustLine
if needed. The change is modeled after offer crossing. And,
similar to offer crossing, cashing a check allows an account to
exceed its trust line limit.
The following changes were made:
- Removed dependency on template defined in beast detail namespace.
- Removed Section::find() method which had an obsolete interface.
- Made Section::get<>() easier to use for the common case of
retrieving a std::string. The revised get() method replaces old
calls to Section::find().
- Provided a default template parameter to free function
get<>(Section config, std::string name) so it stays similar to
Section::get<>().
Then the rest of the code was adapted to these changes.
- Calls to Section::find() were replaced with calls to Section::get.
- Unnecessary get<std::string>() arguments were reduced to get().
These changes dug up an interesting artifact in the SHAMap unit
tests. I'm not sure why the tests were working before, but there
was a problem with the case of a Section key. The unit test is
fixed.
The `[node_size]` configuration parameter is used to tune various
parameters based on the hardware that the code is running on. The
parameter can take five distinct values: `tiny`, `small`, `medium`,
`large` and `huge`.
The default value in the code is `tiny` but the default configuration
file sets the value to `medium`. This commit attempts to detect the
amount of RAM on the system and adjusts the node size default value
based on the amount of RAM and the number of hardware execution
threads on the system.
The decision matrix currently used is:
| | 1 | 2 or 3 | ≥ 4 |
|:-------:|:----:|:------:|:------:|
| > ~8GB | tiny | tiny | tiny |
| > ~12GB | tiny | small | small |
| > ~16GB | tiny | small | medium |
| > ~24GB | tiny | small | large |
| > ~32GB | tiny | small | huge |
Some systems exclude memory reserved by the the hardware, the kernel
or the underlying hypervisor so the automatic detection code may end
up determining the node_size to be one less than "appropriate" given
the above table.
The detection algorithm is simplistic and does not take into account
other relevant factors. Therefore, for production-quality servers it
is recommended that server operators examine the system holistically
and determine what the appropriate size is instead of relying on the
automatic detection code.
To aid server operators, the node size will now be reported in the
`server_info` API as `node_size` when the command is invoked in
'admin' mode.
A recent version of clang notes a number of places in range
for loops where the code base was making unnecessary copies
or using const lvalue references to extend lifetimes. This
fixes the places that clang identified.
* Create SQLite database for mapping transaction IDs to shard indexes
* Create SQLite database for mapping ledger hashes to shard indexes
* Create additional test cases for the shard database
* load_factor was missing from server_info when the server was running in
reporting mode. Now, the reporting mode server calls server_info on the p2p
node, and propagates the load_factor back to the client.
In order to effectively mitigate CVE-2021-3499 even when compiling
against versions of OpenSSL prior to 1.1.1k, this commit:
1) requires use of TLS 1.2 or later. Note that both TLS 1.0 and
TLS 1.1 have been officially deprecated for over a year.
2) disables renegotiation support for TLS 1.2 connections.
Lastly, this commit also changes the default list of ciphers that
the server offers, limiting it only to ciphers that are part of
TLS 1.2.
The `tx` command supports output in both "text" and "binary" modes,
controlled by the binary flag. For more details on the command and
the possible arguments, please see: https://xrpl.org/tx.html.
The existing handler would incorrectly deal with metadata when in
binary mode. This commit corrects this issue, ensuring that the
metadata is properly encoded, depending on the mode.
Typically, an RPC response contains a `result` field, which
contains details about the operation performed. For ease of
parsing, forwarded responses must look like a non-forwarded
response.
In some instances the response was incorrectly composed, so
that the actual `result` object would be encapsulated by an
outer `result` object, breaking existing code.
This commit, addresses this issue and correctly "folds" the
`result` field, ensuring a consistent schema for responses.
* Add instructions to the workflow at the point where the failure would
occur, which is where someone experiencing a failure is most likely to
look. To keep things simple, the instructions are always printed. The
assumption is that if the job succeeds, nobody is likely to look
anyway.
* Provides the diff of the failure as an artifact, so the user can apply
it directly to their repo.
* Also update the levelization/README.md to clarify the levels a little
bit.
This updates the build process to use the local Artifactory server as a docker image cache to avoid being rate limited by docker hub during the build process.
While most of the code associated with secp256k1 operations had
been migrated to libsecp256k1, the deterministic key derivation
code was still using calls to OpenSSL.
If merged, this commit replaces the OpenSSL-based routines with
new libsecp256k1-based implementations. No functional change is
expected and the change should be transparent.
This commit also removes several support classes and utility
functions that wrapped or adapted various OpenSSL types that
are no longer needed.
A tip of the hat to the original author of this truly superb
library, Dr. Pieter Wuille, and to all other contributors.
This commit expands the detection capabilities of the Byzantine
validation detector. Prior to this commit, only validators that
were on a server's UNL were monitored. Now, all the validations
that a server receives are passed through the detector.
The existing class offered several constructors which were mostly
unnecessary. This commit eliminates all existing constructors and
introduces a single new one, taking a `Slice`.
The internal buffer is switched from `std::vector` to `Buffer` to
save a minimum of 8 bytes (plus the buffer slack that is inherent
in `std::vector`) per SHAMapItem instance.
Add support to allow multiple indepedent nodes to produce a binary identical
shard for a given range of ledgers. The advantage is that servers can use
content-addressable storage, and can more efficiently retrieve shards by
downloading from multiple peers at once and then verifying the integrity of
a shard by cross-checking its checksum with the checksum other servers report.
Before this change any non-zero Sequence field was handled as
a non-ticketed transaction, even if a TicketSequence was
present. We learned that this could lead to user confusion.
So the rules are tightened up.
Now if any transaction contains both a non-zero Sequence
field and a TicketSequence field then that transaction
returns a temSEQ_AND_TICKET error code.
The (deprecated) "sign" and "submit" RPC commands are tuned
up so they auto-insert a Sequence field of zero if they see
a TicketSequence in the transaction.
No amendment is needed because this change is going into
the first release that supports the TicketBatch amendment.
* Fix bug where incorrect max amount was set for XRP
* Fix bug where incorrect source currencies were set when XRP was the dst and a
sendmax amount was set
The existing code that deserialized an STAmount was sub-optimal and performed
poorly. In some rare cases the operation could result in otherwise valid
serialized amounts overflowing during deserialization. This commit will help
detect error conditions more quickly and eliminate the problematic corner cases.
* Use theoretical quality to order the strands
* Do not use strands below the user specified quality limit
* Stop exploring strands (at the current quality iteration) once any strand is non-dry
The previous error description was focused on keys that are too long,
but this error can occur if the key is too short or does not contain
the correct prefix.
* Add a new operating mode to rippled called reporting mode
* Add ETL mechanism for a reporting node to extract data from a p2p node
* Add new gRPC methods to faciliate ETL
* Use Postgres in place of SQLite in reporting mode
* Add Cassandra as a nodestore option
* Update logic of RPC handlers when running in reporting mode
* Add ability to forward RPCs to a p2p node
- The changes to manifest relaying introduced with commit f74b469e68
will cause newly accepted manifests to be sent back to the peer from
which they were received. This no longer happens: a newly accepted
manifest is never sent back to the peer we received it from.
- When encountering a manifest without a domain set, the `manifest` and
`validator_info` commands would include an empty string as the domain
associated with the manifest. This no longer happens: if a domain is
not present, the `domain` field will not be.
The existing code attempts to validate the provided node public key
using a function that assumes that the encoded public key is for an
account. This causes the parsing to fail.
This commit fixes#3317 by letting the caller specify the type of
the public key being checked.
The manifest relay code would only ever relay manifests from validators
on a server's UNL which means that the manifests of validators that are
not broadly trusted can fail to propagate across the network, which can
make it difficult to detect and track such validators.
This commit, if merged, propagates all manifests on a best-effort basis
resulting in broader availability of manifests on the network and avoid
the need to introduce on-ledger manifest storage or to establish one or
more manifest repositories.
- Add validation/proposal reduce-relay feature negotiation to
the handshake
- Make squelch duration proportional to a number of peers that
can be squelched
- Refactor makeRequest()/makeResponse() to facilitate handshake
unit-testing
- Fix compression enable flag for inbound peer
- Fix compression algorithm parsing in the header parser
- Fix squelch duration in onMessage(TMSquelch)
This commit fixes 3624, fixes 3639 and fixes 3641
Support for 'out-of-sequence' transaction execution was introduced
in commit 7724cca384.
The changes in that commit were gated under a feature but there was
no corresponding amendment introduced that would allow the network
to vote on this amendment.
This commit introduces 'TicketBatch' amendment as the amendment
that is associated with the tickets feature. If the amendment is
enabled, it will activate support for tickets.
This commit also removes several workarounds that are no longer
needed in unit tests.
* Markdown explanation of what levelization is, the intended levels, as
well as the process used to determine dependencies
* Shell script finds all dependencies, groups them, and finds cyclic
dependencies and maps out non-cyclic dependencies.
* Github job to run the script and fail if anything changes. Should
catch introduction of new dependencies and new problems. Will also
detect changes if problems or dependencies are removed.
* Creates a version 2 of the UNL file format allowing publishers to
pre-publish the next UNL while the current one is still valid.
* Version 1 of the UNL file format is still valid and backward
compatible.
* Also causes rippled to lock down if it has no valid UNLs, similar to
being amendment blocked, except reversible.
* Resolves#3548
* Resolves#3470
* Move all the vcpkg windows dependency installations into one step.
* Move the unmodified `before_install` step above the matrix to improve
readability, because this step runs before any of the matrix steps.
Due to some quirky emergent behavior, the server can't really begin
synching until twice the default close time resolution of the genesis
ledger, which is 30 seconds, has passed. In effect, this causes a one
minute delay.
This commit adjusts the default close time resolution down to the
minimum allowed resoluion of 10 seconds, so the corresponding delay
is reduced by 67% down to 20 seconds. This should be enough time to
ensure the server has reasonable connectivity without unduly delaying
initial synch times.
This comment explains this patch and the associated patches
that should be folded into it. This paragraph should be removed
when the patches are folded after review.
This change significantly improves ledger sync and fetch
times while reducing memory consumption. The change affects
the code from that begins with SHAMap::getMissingNodes and runs
through to Database::threadEntry.
The existing code issues a number of async fetches which are then
handed off to the Database's pool of read threads to execute.
The results of each read are placed in the Database's positive
and negative caches. The caller waits for all reads to complete
and then retrieves the results out of these caches.
Among other issues, this means that the results of the first read
cannot be processed until the last read completes. Additionally,
all the results must sit in memory.
This patch changes the behavior so that each read operation has a
completion handler associated with it. The completion of the read
calls the handler, allowing the results of each read to be
processed as it completes. As this was the only reason the
negative and positive caches were needed, they can now be removed.
The read generation code is also no longer needed and is removed.
The batch fetch logic was never implemented or supported and is
removed.
gcc's implementation of `prm::synchronized_pool_resource` showed
extremely poor performance compared with
`boost::synchronized_pool_resouece`. Boost's implementation of pmr is
now used in all cases (previously it was only used when a standard
lib, like clang's, lacked an implementation of pmr).
This patch also makes a minor change where inner nodes are constructed
with sparse arrays, unless "dense" is explicitly requested.
Prior to this commit, the amendments that a server would vote in support
of or against could be configured both via the configuration file and
via the command line "feature" command. Changes made in the configuration
file would only be loaded once at server startup and changes made via the
command line take effect immediately but are not persisted across
restarts.
This commit deprecates management of amendments via the configuration
file and stores the relevant information in the `wallet.db` database
file.
1. On startup, the new code parses the configuration file.
2. If the `[veto_amendments]` or `[amendments]` sections are present,
we check if the `FeatureVotes` table is present in `wallet.db`.
3. If it is not, we create the `FeatureVotes` table and transfer the
settings from the config file.
4. Proceed normally but only reference the `FeatureVotes` table instead
of the config file.
5. Warns if the voting table already exists in `wallet.db` and there
exists voting sections in the config file. The config file is ignored
in this case.
This change addresses & closes#3366
* Found several functions called under lock that take a lock. Refactor
to require a lock as a parameter instead.
* Found several functions called under lock that don't take a lock, but
should. Refactored those as well to require a lock as a parameter.
Unit tests are counting test failures, process crashes, and process exit code
failures in the count. Since a failing tests causes the process exit code to
return failure, we get extra counts. This patch removes process exit code
failures from the count.
ReadViewFwdRange was storing a cached `end_` iterator that was lazily
created in an iterators `end()` function. When the cache is empty, and
the range it iterated from multiple threads, this creates a race
condition.
This change has performance consequences for "old style" for loops.
For example:
```
// don't do this
for(auto i = tx_range.begin(); i != tx_range.end(); ++i)
```
Can call the now expensive `end()` function more often than needed.
Range-based for loop (I.e. `for(auto const& t : tx_range)`) should be
used instead.
- Under some conditions, comparing `ReadViewFwdRange::iterators`
for equality could derefence an empty `std::unique_ptr` which
will result in a crash.
- Misuse of the `equal` API could result in a `std::bad_cast`
exception being thrown from when iterating transactions or
SLEs from the `OpenView`, `RawStateTable` and `Ledger` classes.
A large percentage of inner nodes only store a small number of children. Memory
can be saved by storing the inner node's children in sparse arrays. Measurements
show that on average a typical SHAMap's inner nodes can be stored using only 25%
of the original space.
This commit combines a number of cleanups, targeting both the
code structure and the code logic. Large changes include:
- Using more strongly-typed classes for SHAMap nodes, instead of relying
on runtime-time detection of class types. This change saves 16 bytes
of memory per node.
- Improving the interface of SHAMap::addGiveItem and SHAMap::addItem to
avoid the need for passing two bool arguments.
- Documenting the "copy-on-write" semantics that SHAMap uses to
efficiently track changes in individual nodes.
- Removing unused code and simplifying several APIs.
- Improving function naming.
- Simplify and consolidate code for parsing hex input.
- Replace beast::endian::order with boost::endian::order.
- Simplify CountedObject code.
- Remove pre-C++17 workarounds in favor of C++17 based solutions.
- Improve `base_uint` and simplify its hex-parsing interface by
consolidating the `SexHex` and `SetHexExact` methods into one
API: `parseHex` which forces callers to verify the result of
the operation; as a result some public-facing API endpoints
may now return errors when passed values that were previously
accepted.
- Remove the simple fallback implementations of SHA2 and RIPEMD
introduced to reduce our dependency on OpenSSL. The code is
slow and rarely, if ever, exercised and we rely on OpenSSL
functionality for Boost.ASIO as well.
- Provide separate functions for serializing depending on whether
one wants a "wire" version of a node, or one suitable for hashing.
- Remove unused functions
The existing SHAMapNodeID object has both a valid and an invalid state
and requirs callers to verify the state of an instance prior to using
it. A simple set of changes removes that restriction and ensures that
all instances are valid, making the code more robust.
This change also:
1. Introduces a new function to construct a SHAMapNodeID from a
serialized blob; and
2. Reduces the amount of constructors the class exposes.
- Limit the lifetime of a buffer that was only used in the early
phases of peer connection establishment but which lived on as
long as the peer was active.
- Cache the message used to transfer manifests, so it can be reused
instead of recreated for every peer connection.
- Improve the reading of partial messages by passing a hint to the
I/O layer if the number of bytes needed to complete the message
is known.
The existing code issues a PING to each peer every 8 seconds. While
frequent PINGs allow us to estimate a peer's latency with a high
degree of accuracy, this "inter-server polka dance" is inefficient
and not useful. This commit, if merged, reduces the PING frequency
to once every 60 seconds.
Additionally, this commit simplifies the PING handling logic and
merges the code used to check and disconnect peers which fail to
track the network directly into the timer callback.
When evaluating the fitness and usefulness of an outbound peer, the code
would incorrectly calculate the amount of time that the peer spent in
a non-useful state.
This commit, if merged, corrects the calculation and makes the timeout
values configurable by server operators.
Two new options are introduced in the 'overlay' stanza of the config
file. The default values, in seconds, are:
[overlay]
max_unknown_time = 600
max_diverged_time = 300
This commit replaces the `peers_max` configuration element which had
a predetermined split between incoming and outgoing connections with
two new configuration options, `peers_in_max` and `peers_out_max`,
which server operators can use to explicitly control the number of
incoming and outgoing peer slots.
There have been cases in the past where SFields have been defined
in such a way that they did not follow our conventions. In
particular, the string representation of an SField should match
the in-code name of the SField.
This change leverages the preprocessor to encourage SFields to
be properly constructed.
The suffixes of SField types are changed to be the same as
the suffixes of corresponding SerializedTypeIDs. This allows
The preprocessor to match types using simple name pasting.
Since the string representation of the SField is part of our
stable API, the name of sfPayChannel was changed to sfChannel.
This change allows sfChannel to follow our conventions while
making no changes to our external API.
* Remove DinD Service from container build template
DinD has changed how it works on GItLab due to recent docker changes such that the service no long needs to be called so long as the runner is being run on a `docker-X` tagged machine.
* refactor for docker service on normal node
* If multiple transactions are queued for the account, change the
account's sequence number in a temporary view before processing the
transaction.
* Adds a new "at()" interface to STObject which is identical to the
operator[], but easier to write and read when dealing with ptrs.
* Split the TxQ tests into two suites to speed up parallel run times.
This commit introduces a new configuration option that server
operators can set. The value is communicated to other servers
and is also reported via the `server_info` API.
The value is meant to allow third-party applications or tools
to group servers together. For example, a tool that visualizes
the network's topology can group servers together.
Similar to the "Domain" field in validator manifests, an operator
can claim any domain. Prior to relying on the value returned, the
domain should be verified by retrieving the xrp-ledger.toml file
from the domain and looking for the server's public key in the
`nodes` array.
* Increases hard-coded number of parallel unit test processes for
Windows and MacOS builds from 1 to 2.
* Reduces Travis job time to well under the timeout value of 1.5 hours.
* Continue using a hard-coded value rather than `nprocs` because higher
values cause some jobs to run out of memory.
* Jobs with no unit tests are counted as failures. Resolves#3474
* Crashed processes are counted as failures. Resolves#3600
* Any tests specified on the command line test do not have matching
suites are counted as failures.
* Remove unused CI manual test.
When processing the `tx` command, we will now load both the transaction
and its metadata directly from SQLite.
Previously the `tx` RPC call was querying SQLite for the transaction
and then separately querying the key-value store for the metadata.
Support for IPv6 messages was added with commit 08382d866b
and version 1.1.0. No peer presently connected to the network in a useful capacity fails
to understand v2 messages.
This commit removes the code that generates and processes v1 messages and deletes legacy
messages from the protocol buffer definition file.
Use C++17 constant expressions to calculate the inverse
alphabet map at compile time instead of at runtime.
Remove support for encoding & decoding tokens using the
Bitcoin alphabet.
The job queue can impose limits of how many jobs of a particular
type can be queued.
This commit makes the previously hard-coded limit associated with
transactions configurable by the server's operator. Servers that
have increased memory capacity or which expect to see an influx
of transactions can increase the number of transactions their
server will be able to queue.
This commit fixes#3556.
The "/vl" HTTP endpoint can be used to request a particular
UNL from a rippled instance.
This commit, if merged, includes the public key of the requested
list in the response.
This commit fixes#3392
* Distinguish between recent and historical shards
* Allow multiple storage paths for historical shards
* Add documentation for this feature
* Add unit tests
Some RPC commands return `ledger_index` as a quoted numeric
string. This change allows the returned value to be directly
copied and used for follow-on RPC commands.
This commit fixes#3533
When attempting to load a validator list from a configured
site, attempt to reuse the last IP that was successfully
used if that IP is still present in the DNS response.
Otherwise, randomly select an IP address from the list of
IPs provided by the DNS system.
This commit fixes#3494.
With few exceptions, servers will typically receive multiple copies
of any given message from its directly connected peers. For servers
with several peers this can impact the processing latency and force
it to do redundant work. Proposal and validation messages are often
relayed with extremely high redundancy.
This commit, if merged, introduces experimental code that attempts
to optimize the relaying of proposals and validations by allowing
servers to instruct their peers to "squelch" delivery of selected
proposals and validations. Servers making squelching decisions by
a process that evaluates the fitness and performance of a given
server and randomly selecting a subset of the best candidates.
The experimental code is presently disabled and must be explicitly
enabled by server operators that wish to test it.
Tickets are a mechanism to allow for the "out-of-order" execution of
transactions on the XRP Ledger.
This commit, if merged, reworks the existing support for tickets and
introduces support for 'ticket batching', completing the feature set
needed for tickets.
The code is gated under the newly-introduced `TicketBatch` amendment
and the `Tickets` amendment, which is not presently active on the
network, is being removed.
The specification for this change can be found at:
https://github.com/xrp-community/standards-drafts/issues/16
Commit 4dc08f8202 introduced support for
deterministic shards, which makes the sharding functionality provided
by rippled more useful.
After merging, several opportunities for further improvements to the
deterministic sharding implementation were identified and a significant
increase int memory usage during shard finalization was detected.
Because of these issues, the commit is being reverted and the feature is
being rolled back. It will be reintroduced in a future release.
* Builds Windows dependencies first.
* Builds ALL OSs in the last stage.
* Fix the MacOS builds.
* Windows dependency stages are allowed to fail so ALL configurations will
attempt to build. Windows builds will probably fail if dependencies fail
(caching may allow them to succeed), but they will at least be attempted.
* Remove broken AppVeyor config file, so it stops trying.
The checkpointer class had assumed that the database would exist for the
lifetime of the application. This is no long true. These changes resolve bugs
involving dangling pointers.
There was a race condition in `on_accept` where the object's destructor
could run while `on_accept` was called.
This patch ensures that if `on_accept` is called then the object remains
valid for the duration of the call.
* Fixes#3486
* load factor computation normalized by load_base.
* last validated ledger age set to -1 while syncing.
* Return status changed:
* healthy -> ok
* warning -> service_unavailable
* critical -> internal_server_error
This change can help improve the liveness of the network during periods of network
instability, by allowing the network to track which validators are presently not online
and to disregard them for the purposes of quorum calculations.
If the 'HardenedValidations' amendment is enabled, this commit will
track the version of the software that validators embed in their
validations.
If a server notices that at least 60% of the validators on its UNL
are running a newer version than it is running, it will periodically
print an informational message, reminding the operator to check for
update.
The tecUNFUNDED code is actively used when attempting to create payment
channels; the messages incorrectly list it as deprecated.
Meanwhile, the tecUNFUNDED_ADD code actually is an unused legacy code,
dating back to when there was a WalletAdd transactor. The terLAST and
terFUNDS_SPENT codes are also unused legacy codes.
Engine result messages are not part of the binary format and are
documented as subject to change without notice, so this should not
require an amendment nor a new API version.
Align error code table for human readability.
The amendment was partially complete, included no functional code
and, even if activated, it would result in no changes to transaction
proessing. Despite this, removing the amendment is the prudent course
of action and avoids the possibility of an accidental activation.
If additional cryptoconditions are implemented, they will be each
assigned a new, unique amendment code.
This commit, if merged, adds support to allow multiple indepedent nodes to
produce a binary identical shard for a given range of ledgers. The advantage
is that servers can use content-addressable storage, and can more efficiently
retrieve shards by downloading from multiple peers at once and then verifying
the integrity of a shard by cross-checking its checksum with the checksum
other servers report.
* Document delete_batch, back_off_milliseconds, age_threshold_seconds.
* Convert those time values to chrono types.
* Fix bug that ignored age_threshold_seconds.
* Add a "recovery buffer" to the config that gives the node a chance to
recover before aborting online delete.
* Add begin/end log messages around the SQL queries.
* Add a new configuration section: [sqlite] to allow tuning the sqlite
database operations. Ignored on full/large history servers.
* Update documentation of [node_db] and [sqlite] in the
rippled-example.cfg file.
Resolves#3321
* The amendment ballot counting code contained a minor technical
flaw, caused by the use of integer arithmetic and rounding
semantics, that could allow amendments to reach majority with
slightly less than 80% support. This commit introduces an
amendment which, if enabled, will ensure that activation
requires at least 80% support.
* This commit also introduces a configuration option to adjust
the amendment activation hysteresis. This option is useful on
test networks, but should not be used on the main network as
is a network-wide consensus parameter that should not be
changed on a per-server basis; doing so can result in a
hard-fork.
Fixes#3396
Work on a version 2 of the XRP Network API has begun. The new
API returns:
* `notSynced` in place of `noClosed`, `noCurrent`, and `noNetwork`;
* `invalidParams` in place of `lgrIdxInvalid`.
The new version 2 API cannot be selected yet, as it remains a work
in progress.
Fixes#3269
If a port number is not specified in the [ips] or [ips_fixed]
blocks, automatically add the new default peer port which was
registered with IANA: 2459. Also use 2459 if no port is specified
with manually using the `connect` command; previously it was
using 6561, which could have resulted in spurious failures.
This commit, if merged, fixes#2861.
* Gives a summary of the health of the node:
Healthy, Warning, or Critical
* Last validated ledger age:
<7s is Healthy,
7s to 20s is Warning
> 20s is Critcal
* If amendment blocked, Critical
* Number of peers:
> 7 is Healthy
1 to 7 is Warning
0 is Critical
* server state:
One of full, validating or proposing is Healthy
One of syncing, tracking or connected is Warning
All other states are Critical
* load factor:
<= 100 is Healthy
101 to 999 is Warning
>= 1000 is Critical
* If not Healthy, info field contains data that is considered not
Healthy.
Fixes: #2809
Commit e257a22 introduced changes in the logic used to acquire historical
ledgers. The logic could cause historical ledgers to be acquired only since
the last online deletion interval instead of the configured value to allow
deletion.
* Make sure variables are always initialized
* Use lround instead of adding .5 and casting
* Remove some unneeded vars
* Check for null before calling strcmp
* Remove redundant if conditions
* Remove make_TxQ factory function
* Improve documentation
* Make the ShardArchiveHandler rather than the DatabaseShardImp perform
LastLedgerHash verification for downloaded shards
* Remove ShardArchiveHandler's singleton implementation and make it an
Application member
* Have the Application invoke ShardArchiveHandler initialization
instead of clients
* Add RecoveryHandler as a ShardArchiveHandler derived class
* Improve commenting
* Add documentation for shard validation
* Retrieve last ledger hash for imported shards
* Verify the last ledger hash in Shard::finalize
* Limit last ledger hash retrieval attempts for imported shards
* Use a common function for removing failed shards
* Add new ShardInfo::State for imported shards
Identifiers for retired amendments should not generally be used
in the codebase.
This commit reduces their visibility down to one translation
unit and marks them as unused and deprecated to prevent
accidental reuse.
In deciding whether to relay a proposal or validation, a server would
consider whether it was issued by a validator on that server's UNL.
While both trusted proposals and validations were always relayed,
the code prioritized relaying of untrusted proposals over untrusted
validations. While not technically incorrect, validations are
generally more "valuable" because they are required during the
consensus process, whereas proposals are not, strictly, required.
The commit introduces two new configuration options, allowing server
operators to fine-tune the relaying behavior:
The `[relay_proposals]` option controls the relaying behavior for
proposals received by this server. It has two settings: "trusted"
and "all" and the default is "trusted".
The `[relay_validations]` options controls the relaying behavior for
validations received by this server. It has two settings: "trusted"
and "all" and the default is "all".
This change does not require an amendment as it does not affect
transaction processing.
The sfLedgerSequence field is designated as optional in the object
template but it is effectively required and validations which do not
include it were, correctly, rejected.
This commit migrates the check outside of the peer code and into the
constructor used for validations being deserialized for the network.
Furthermore, the code will generate an error if a validation that is
generated by a server does not include the field.
The existing code used std::deque along with a size check to constrain the
size of a buffer and, effectively, "hand rolled" a circular buffer. This
change simply migrates directly to boost::circular_buffer.
The unit test now verifies that if an account is not present in the
starting account_tx ledger, account_tx still iterates down and finds
the transaction that deletes the account (and earlier transactions).
This commit introduces no functional changes but cleans up the
code and shrinks the surface area by removing dead and unused
code, leveraging std:: alternatives to hand-rolled code and
improving comments and documentation.
The script, when invoked by a server operator can collect information
useful for debugging, while attempting to redact potentially sensitive
data.
It contained no explanation or other exposition to allow people who
look at the file but aren't familiar with shell scripts to understand
its purpose.
The built-in watchdog is simplistic and can, sometimes, cause problems
especially on systems that have the ability to automatically start and
monitor processes.
This commit removes the sustain system entirely, changes the handling
of the SIGTERM signal to properly terminate the process and improves
the error message reported to the user when the command line used to
start `rippled` is incorrect and malformed.
Entries in the ledger are located using 256-bit locators. The locators
are calculated using a wide range of parameters specific to the entry
whose locator we are calculating (e.g. an account's locator is derived
from the account's address, whereas the locator for an offer is derived
from the account and the offer sequence.)
Keylets enhance type safety during lookup and make the code more robust,
so this commit removes most of the earlier code, which used naked
uint256 values.
This commit removes obsolete comments, dead or no longer useful
code, and workarounds for several issues that were present in older
compilers that we no longer support.
Specifically:
- It improves the transaction metadata handling class, simplifying
its use and making it less error-prone.
- It reduces the footprint of the Serializer class by consolidating
code and leveraging templates.
- It cleanups the ST* class hierarchy, removing dead code, improving
and consolidating code to reduce complexity and code duplication.
- It shores up the handling of currency codes and the conversation
between 160-bit currency codes and their string representation.
- It migrates beast::secure_erase to the ripple namespace and uses
a call to OpenSSL_cleanse instead of the custom implementation.
A deliberately malformed token can cause the server to crash during
startup. This is not remotely exploitable and would require someone
with access to the configuration file of the server to make changes
and then restart the server.
Acknowledgements:
Guido Vranken for responsibly disclosing this issue.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to
responsibly disclose any issues they may find.
Ripple is generously sponsoring a bug bounty program for the
rippled project. For more information please visit:
https://ripple.com/bug-bounty
Currently there is no mechanism for a validator to report the
version of the software it is currently running. Such reports
can be useful for those who are developing network monitoring
dashboards and server operators in general.
This commit, if merged, defines an encoding scheme to encode
a version string into a 64-bit unsigned integer and adds an
additional optional field to validations.
This commit piggybacks on "HardenedValidations" amendment to
determine whether version information should be propagated
or not.
The general encoding scheme is:
XXXXXXXX-XXXXXXXX-YYYYYYYY-YYYYYYYY-YYYYYYYY-YYYYYYYY-YYYYYYYY-YYYYYYYY
X: 16 bits identifying the particular implementation
Y: 48 bits of data specific to the implementation
The rippled-specific format (implementation ID is: 0x18 0x3B) is:
00011000-00111011-MMMMMMMM-mmmmmmmm-pppppppp-TTNNNNNN-00000000-00000000
M: 8-bit major version (0-255)
m: 8-bit minor version (0-255)
p: 8-bit patch version (0-255)
T: 11 if neither an RC nor a beta
10 if an RC
01 if a beta
N: 6-bit rc/beta number (1-63)
This commit introduces the "HardenedValidations" amendment which,
if enabled, allows validators to include additional information in
their validations that can increase the robustness of consensus.
Specifically, the commit introduces a new optional field that can
be set in validation messages can be used to attest to the hash of
the latest ledger that a validator considers to be fully validated.
Additionally, the commit leverages the previously introduced "cookie"
field to improve the robustness of the network by making it possible
for servers to automatically detect accidental misconfiguration which
results in two or more validators using the same validation key.
- Add missing `#include` in `ripple/core/JobTypeInfo.h`
- Protect version string from clang-format in
`ripple/protocol/impl/BuildInfo.cpp`.
`Builds/CMake/RippledVersion.cmake` searches for this line by pattern.
Existing per-thread PRNGs are individually initialized using calls
to std::random_device.
If merged, this commit will use a single PRNG, initialized from
std::random_device on startup, to seed the thread-specific PRNGs.
Acknowledgements:
Thomas Snider, who reported this issue to Ripple on April 8, 2020.
Historically strand re-execute log messages have been treated as
errors. However in the vast majority of cases these log messages
are caused by well understood mechanics in the payment engine.
So usually these log messages should be treated as warnings.
The automated build system only builds packages signed with a list of
approved keys. This is a security measure to prevent someone who gains
push access to the repository from producing potentially malicious
packages that are signed by Ripple's trusted private keys.
Moving this list to the new location makes it easy to add and delete
new keys to the list.
* scoped_lock is now a std name with subtly different semantics
compared to lock_guard. Namely it can be used to lock 0 or
more mutexes. This is valuable, but can also be accidentally
used to lock 0 mutexes when 1 was intended, creating a
run-time error.
Therefore, if and when we use scoped_lock, extra care needs to
be taken in reviewing that code to ensure it doesn't
accidentally lock 0 mutexes when 1 was intended. To aid in
such careful reviewing, the use of the name scoped_lock should
be limited to those cases where the number of mutexes is not
exactly one.
* canonicalize_replace_cache
* canonicalize_replace_client
Now it is clear at the call site that if there are
duplicate copies of the data between the cache and
the caller, which copy gets replaced.
Additionally data parameter is now const-correct.
If it is not going to be replaced (canonicalize_replace_cache),
then the shared_ptr to the client data is const.
* The [network_id] option allows three string values:
- main: the XRP Ledger
- testnet: the Testnet operated by Ripple.
- devnet: the development test network operated by Ripple.
* Peers negotiate compression via HTTP Header "X-Offer-Compression: lz4"
* Messages greater than 70 bytes and protocol type messages MANIFESTS,
ENDPOINTS, TRANSACTION, GET_LEDGER, LEDGER_DATA, GET_OBJECT,
and VALIDATORLIST are compressed
* If the compressed message is larger than the uncompressed message
then the uncompressed message is sent
* Compression flag and the compression algorithm type are included
in the message header
* Only LZ4 block compression is currently supported
The payment engine restricts payment paths so two steps do not input the
same Currency/Issuer or output the same Currency/Issuer. This check was
skipped when the path started or ended with XRP. An example of a path
that was incorrectly accepted was: XRP -> //USD -> //XRP -> EUR
This patch enables the path loop check for paths that start or end with
XRP.
* Make ShardArchiveHandler a singleton.
* Add state database for ShardArchiveHandler.
* Use temporary database for SSLHTTPDownloader downloads.
* Make ShardArchiveHandler a Stoppable class.
* Automatically resume interrupted downloads at server start.
* Reduce lock scope on all public functions
* Use TaskQueue to process shard finalization in separate thread
* Store shard last ledger hash and other info in backend
* Use temp SQLite DB versus control file when acquiring
* Remove boost serialization from cmake files
When computing rates for offers, an STAmount's value can be out of can be out of
range (before canonicalizing). There was an assert that could incorrectly fire
in some cases. This patch removes that assert.
The newest MSVC 19.25.28610.4 does not build rocksdb. During the
Travis CI Windows job, the vs_BuildTools.exe automatically
downloads the newest version of the compiler. This fix forces the
install of MSVC 19.24.28314.0 to build rocksdb.
The fix1781 amendment was incorrectly introduced during conflict
resolution and support for it is not included at this time. This commit
removes the definition of the amendment identifier.
A review of the lag ratchet code revealed that we were using
the long-term master public keys of trusted validators, when
we should have been using the ephemeral public keys instead.
As a result, the lag ratchet code would be effectively
inoperable.
- Add support for all transaction types and ledger object types to gRPC
implementation of tx and account_tx.
- Create common handlers for tx and account_tx.
- Remove mutex and abort() from gRPC server. JobQueue is stopped before
gRPC server, with all coroutines executed to completion, so no need for
synchronization.
* Whenever a node downloads a new VL, send it to all peers that
haven't already sent or received it. It also saves it to the
database_dir as a Json text file named "cache." plus the public key of
the list signer. Any files that exist for public keys provided in
[validator_list_keys] will be loaded and processed if any download
from [validator_list_sites] fails or no [validator_list_sites] are
configured.
* Whenever a node receives a broadcast VL message, it treats it as if
it had downloaded it on it's own, broadcasting to other peers as
described above.
* Because nodes normally download the VL once every 5 minutes, a single
node downloading a VL with an updated sequence number could
potentially propagate across a large part of a well-connected network
before any other nodes attempt to download, decreasing the amount of
time that different parts of the network are using different VLs.
* Send all of our current valid VLs to new peers on connection.
This is probably the "noisiest" part of this change, but will give
poorly connected or poorly networked nodes the best chance of syncing
quickly. Nodes which have no http(s) access configured or available
can get a VL with no extra effort.
* Requests on the peer port to the /vl/<pubkey> endpoint will return
that VL in the same JSON format as is used to download now, IF the
node trusts and has a valid instance of that VL.
* Upgrade protocol version to 2.1. VLs will only be sent to 2.1 and
higher nodes.
* Resolves#2953
* Example: gcc.Debug will use the the default version of gcc installed on the
system. gcc-9.Debug will use version 9, regardless of the default. This will
be most useful when the default is older than required or desired.
* When an unknown amendment reaches majority, log an error-level
message, and return a `warnings` array on all successful
admin-level RPC calls to `server_info` and `server_state` with
a message describing the problem, and the expected deadline.
* In addition to the `amendment_blocked` flag returned by
`server_info` and `server_state`, return a warning with a more
verbose description when the server is amendment blocked.
* Check on every flag ledger to see if the amendment(s) lose majority.
Logs again if they don't, resumes normal operations if they did.
The intention is to give operators earlier warning that their
instances are in danger of being amendment blocked, which will
hopefully motivate them to update ahead of time.
* update EP and find package requirements
* minor protobuf/libarchive build fixes
* change travis release builds to nounity to
ameliorate vm memory exhaustion.
FIXES: #3223, #3232
* In and Out parameters were swapped when calculating the rate
* In and out qualities were not calculated correctly; use existing functions
to get the qualities
* Added tests to check that theoretical quality matches actual computed quality
* Remove in/out parameter from qualityUpperBound
* Rename an overload of qualityUpperBound to adjustQualityWithFees
* Add fix amendment
STAmount::soTime and soTime2 were time based "amendment like"
switches to control small changes in behavior for STAmount.
soTime2, which was the most recent, was dated Feb 27, 2016.
That's over 3 years ago.
The main reason to retain these soTimes would be to replay
old transactions. The likelihood of needing to replay a
transaction from over three years ago is pretty low. So it
makes sense to remove these soTime values.
In Flow_test the testZeroOutputStep() test is removed. That
test started to fail when the STAmount::soTimes were removed.
I checked with the original author of the test. He said
that the code being tested by that unit test has been removed,
so it makes sense to remove the test. That test is removed.
* use tagged containers for pkg build
* update build images
* continue to build container images in pipeline, but allow
failure (non-block)
* limit travis macos cache
* add vs2019 windows to travis
* remove xcode 9 travis build
* remove clang5/6 from CI and update min version of Clang required in
cmake
* break windows CI build into stages to reduce timeouts
* update datelib
* add if condition to travis builds to allow commit message to limit
builds by platform
FIXES: #2847
* Transactions that are submitted with the fail_hard flag
and that result in any TER code besides tesSUCCESS shall
be neither queued nor held.
[FOLD] Keep tec results out of the open ledger when fail_hard:
* Improve TransactionStatus const correctness, and remove redundant
`local` check
* Check open ledger tx count in fail_hard tests
* Fix some wrapping
* Remove duplicate test
Remove the implicit conversion from int64 to XRPAmount. The motivation for this
was noticing that many calls to `to_string` with an integer parameter type were
calling the wrong `to_string` function. Since the calls were not prefixed with
`std::`, and there is no ADL to call `std::to_string`, this was converting the
int to an `XRPAmount` and calling `to_string(XRPAmount)`.
Since `to_string(XRPAmount)` did the same thing as `to_string(int)` this error
went undetected.
Prior to this commit, the queue and execution times for individual jobs
were reported indepedently and could, potentially, be out of sync. This
change reports both values when either one of the exceeds the reporting
threshold.
It's possible an overloaded job queue is causing false alarms on the deadlock
detector. Log a fatal message after 90s, declare a logic error after 600s.
If merged, this commit will report additional information in the
response to the submit command; this will make it easier for developers
to accurately track the status of transaction submission.
Fixes#2851
Treat all `#` characters in config files as comments (and remove)
*unless* the `#` is immediately preceded by `\`. Write a warning
to log file when trailing comments are found/ignored in the config
to let operators know that the treatment of trailing `#` has changed.
Fixes#3121
The 'network_id' option allows an administrator to specify to which
network they intend a server to connect. Servers can leverage this
information to optimize routing and prune automatically discovered
cross-network connections.
This commit will, if merged:
- add support for the devnet keyword, which corresponds to network ID #2;
- report the network ID, if one is configured, in server_info
The `node_size` configuration option is used to automatically
configure various parameters (cache sizes, timeouts, etc) for
the server.
A previous commit included changes that caused incorrect values
to be returned which can result in sub-optimal performance that
can manifest as difficulty syncing to the network, or increased
disk I/O and/or memory usage. The problem was introduced with
commit 66fad62e66.
This commit, if merged, fixes the code to ensure that the correct
values are returned and introduces a compile-time check to prevent
this issue from reoccurring.
The existing platform detection code was derived from the old Beast
library, which was, itself, derived from JUCE.
This commit removes that code and replaces it with the Boost.Predef
library which defines a consistent set of compiler, architecture,
operating system, library, and other version numbers.
For more on Boost.Predef, please see the Boost documentation. The
documentation for the current version as of this writing is at:
https://www.boost.org/doc/libs/1_71_0/doc/html/predef.html
This commit restructures the HTTP based protocol negotiation that `rippled`
executes and introduces support for negotiation of compression for peer
links which, if implemented, should result in significant bandwidth savings
for some server roles.
This commit also introduces the new `[network_id]` configuration option
that administrators can use to specify which network the server is part of
and intends to join. This makes it possible for servers from different
networks to drop the link early.
The changeset also improves the log messages generated when negotiation
of a peer link upgrade fails. In the past, no useful information would
be logged, making it more difficult for admins to troubleshoot errors.
This commit also fixes RIPD-237 and RIPD-451
* The `tx` command now supports min_ledger and max_ledger fields.
* If the requested transaction isn't found and these fields are
provided, the error response indicates whether or not every
ledger in the the provided range was searched.
This fixes#2924
* adding package signing steps for rpm and deb
* first spike at GPG signing with CI and containers
* refine ubuntu portion
* get correct gpg package version
* adding CentOS support
* fixing errors in installing gpg on ubuntu
* base64 decode the GPG key
* fixing line continuations
* revised package signing, looking for package artifacts
* add dpkg-sig to ubuntu image
* sign all deb packges
* add passphrase to GPG process
* repeat yo slef on dpkg
* sign all the rpm packages too
* install rpm-sign in the CentOS docker image
* loop through rpm files
* no need for PIN on GPG signing
Collecting the returned and expected values in sets only works if there are no
duplicates. The implementation is changed to use sorted vectors to fix this case.
When the Env::AppBundle constructor throws an exception
it still needs to run ~AppBundle(), otherwise the JobQueue
isn't properly shut down. Specifically the JobQueue
can destruct without waiting on outstanding jobs in the
queue.
This change ensures that if Env::AppBundle constructor
throws, Env::AppBundle::~AppBundle() runs.
This fixes the unit test crash exposed by PR #3047.
FIXES: #3106
Different versions of protobuf produce subtly different
results when given invalid message payloads. This leads to
subtly different behavior when we try to deserialize these
invalid messages. As such, we can't tie success to a
particular exception.
The XRP Ledger utilizes an account model. Unlike systems based on a UTXO
model, XRP Ledger accounts are first-class objects. This design choice
allows the XRP Ledger to offer rich functionality, including the ability
to own objects (offers, escrows, checks, signer lists) as well as other
advanced features, such as key rotation and configurable multi-signing
without needing to change a destination address.
The trade-off is that accounts must be stored on ledger. The XRP Ledger
applies reserve requirements, in XRP, to protect the shared global ledger
from growing excessively large as the result of spam or malicious usage.
Prior to this commit, accounts had been permanent objects; once created,
they could never be deleted.
This commit introduces a new amendment "DeletableAccounts" which, if
enabled, will allow account objects to be deleted by executing the new
"AccountDelete" transaction. Any funds remaining in the account will
be transferred to an account specified in the deletion transaction.
The amendment changes the mechanics of account creation; previously
a new account would have an initial sequence number of 1. Accounts
created after the amendment will have an initial sequence number that
is equal to the ledger in which the account was created.
Accounts can only be deleted if they are not associated with any
obligations (like RippleStates, Escrows, or PayChannels) and if the
current ledger sequence number exceeds the account's sequence number
by at least 256 so that, if recreated, the account can be protected
from transaction replay.
* replace boost::beast::detail::iequals with boost::iequals
* replace deprecated `buffers` function with `make_printable`
* replace boost::beast::detail::ascii_tolower with lambda
* add missing includes
The validation stream only reported the ephemeral signing key for validators
which use manifests. This made tracking unnecessarily difficult for clients
processing the data stream.
With this change, the validator's long-term master public key is also
included.
This commit fixes#3005
* Provide proposing validator's master key in the validation stream
subscription JSON responses.
Implement code review changes.
FIXES: #3005
* Explain that Arch/Manjaro/etc. need `-Dstatic=OFF` during the configure step
* move configuration options closer to that step
* separate sub-headers for configuration and build
Different compilers and handling the shadow warning differently. In particular,
some are warning about types being shadowed by variables. Until these can be
resolved the shadow warning is being disabled.
Note: the shadow warning was originally enabled to help with the structured
bindings patch. As that is now complete, it's less important to keep this
warning.
FIXES: #2527
* define custom docker image for travis-linux builds based on
package build image
* add macos builds
* add windows builds (currently allowed to fail)
* improve build and shell scripts as required for the CI envs
* add asio timer latency workaround
* omit several manual tests from TravisCI which cause memory exhaustion
This commit allows server operators to reserve slots for specific
peers (identified by the peer's public node identity) and to make
changes to the reservations while the server is operating.
This commit closes#2938
- Add docker container tags for "latest_BRANCH"
- Prevent different branches from overwriting deb repo artifacts
- Manual approval always required before pushing to prod
The original intent was that RPC error codes were not stable.
But those codes were made available through the API, so some
users came to depend on the code values. This change adapts
to the current state of affairs.
Manifests which are revoked can include ephemeral keys although doing
so does not make sense: a revoked manifest isn't used for signing and
so don't need to define an ephemeral key.
A running instance of the server tracks the number of protocol messages
and the number of bytes it sends and receives.
This commit makes the counters more granular, allowing server operators
to better track and understand bandwidth usage.
* Add construction and assignment from a generic
contiguous container. Both compile-time and run time
safety checks are made to ensure the safety of this
conversion.
* Remove base_uint::copyFrom. The generic copy assignment
operator now does this functionality with enhanced
safety and better syntax.
* Remove construction from and dedendence on Blob.
The generic constructor and assignment now handle this
functionality.
* Fix client code to adhere to this new API.
* Removed the use of fromVoid in PeerImp.cpp as it was
an inappropriate use of this dangerous API. The
generic container constructors do it with enhanced
safety and better syntax.
* Rename data member pn to data_ and make it private.
* Remove constraint from hash_append
* Remove array_type alias
This PR addresses a problem where the server could hang indefinitely
on shutdown. The cause of the problem is the SNTPClock class was not
binding the socket to an endpoint on initialization. This can create
an error sent to the read handler. Unfortunately, the handler ignores
the error, reads again and enters into a loop preventing the
io_service from ever completing.
- Explain how to bind to both IPv4 and IPv6 interfaces
- Provide a hint in the default [port_peer] section
- Do not enable it by default
Note that on Linux, use of '::' and IPv4-mapped IPv6 depends on a sysctl value
setting 'net.ipv6.bindv6only = 0' which seems to be the default on most Linux
distributions.
- Use `std::lock` when grabbing multiple mutexes to ensure consistent
locking order and avoid deadlocks.
- Reduce the scope of the master mutex lock by relesing it prior to
calling setHeartbeatTimer
A tiny input amount to a payment step can cause this step to output zero. For
example, if a previous steps outputs a dust amount of 10^-80, and this step is a
IOU -> XRP offer, the offer may output zero drops. In this case, call the strand
dry. Before this patch, an error would be logged, the strand would be called
dry; in debug mode an assert triggered.
Note, this patch is not transaction breaking, as the caller did not user the ter
code. The caller only checked for success or failuer.
This patch addresses github issue issue reported here:
https://github.com/ripple/rippled/issues/2929
This patch removes calls to several deprecated asio functions.
* `io_service::post` becomes `post` (free function)
* `io_service::work` becomes `executor_work_guard`
* `io_service::wrap` becomes `bind_executor`
* `get_io_context` becomes `get_executor` or `get_executor().context()`
This patch was tested with boost 1.69 and 1.70. The functions
`ripple::get_lowest_layer` and `beast::create_waitable_timer` are required to
handle a breaking difference between these versions. When rippled no longer
needs to support pre 1.70 boost versions, both of these functions may be
removed, and the waitable timer injections may also be removed.
The new parse logic is more strict but handles more cases. If an exception
is thrown, just bail.
* Allow parsing unenclosed IPv6 addresses without port
* Improve string construction
* Reduce nesting levels of code
The XRP Ledger allows an account to authorize a secondary key pair,
called a regular key pair, to sign future transactions, while keeping
the master key pair offline.
The regular key pair can be changed as often as desired, without
requiring other changes on the account.
If merged, this commit corrects a minor technical flaw which would
allow an account holder to specify the master key as the account's
new regular key.
The change is controlled by the `fixMasterKeyAsRegularKey` amendment
which, if enabled, will:
1. Prevent specifying an account's master key as the account's
regular key.
2. Prevent the "Disable Master Key" flag from incorrectly affecting
regular keys.
Before this patch, jtx allowed non-invocable functions to be passed to
operator(). However, these arguments are ignored. This caused erronious code
code such as:
```
env (offer (account_to_test, BTC (250), XRP (1000)),
offers (account_to_test, 1));
```
While it looks like the number of offers are checked, they are not. The `offers`
funclet is never run. While we could modify jtx to make the above code correct,
a cleaner solution is to run post conditions in a `require` statement after a
transasction runs.
At this point all of the jss::* names are defined in the same
file. That file has been named JsonFields.h. That file name
has little to do with either JsonStaticStrings (which is what
jss is short for) or with jss. The file is renamed to jss.h
so the file name better reflects what the file contains.
All includes of that file are fixed. A few include order
issues are tidied up along the way.
Formerly an SOTemplate was default constructed and its elements
added using push_back(). This left open the possibility of a
malformed SOTemplate if adding one of the elements caused a throw.
With this commit the SOTemplate requires an initializer_list of
its elements at construction. Elements may not be added after
construction. With this approach either the SOTemplate is fully
constructed with all of its elements or the constructor throws,
which prevents an invalid SOTemplate from even existing.
This change requires all SOTemplate construction to be adjusted
at the call site. Those changes are also in this commit.
The SOE_Flags enum is also renamed to SOEStyle, which harmonizes
the name with other uses in the code base. SOEStyle elements
are renamed (slightly) to have an "soe" prefix rather than "SOE_".
This heads toward reserving identifiers with all upper case for
macros. The new style also aligns with other prominent enums in
the code base like the collection of TER identifiers.
SOElement is adjusted so it can be stored directly in an STL
container, rather than requiring storage in a unique_ptr.
Correspondingly, unique_ptr usage is removed from both
SOTemplate and KnownFormats.
The new 'Domain' field allows validator operators to associate a domain
name with their manifest in a transparent and independently verifiable
fashion.
It is important to point out that while this system can cryptographically
prove that a particular validator claims to be associated with a domain
it does *NOT* prove that the validator is, actually, associated with that
domain.
Domain owners will have to cryptographically attest to operating particular
validators that claim to be associated with that domain. One option for
doing so would be by making available a file over HTTPS under the domain
being claimed, which is verified separately (e.g. by ensuring that the
certificate used to serve the file matches the domain being claimed) and
which contains the long-term master public keys of validator(s) associated
with that domain.
Credit for an early prototype of this idea goes to GitHub user @cryptobrad
who introduced a PR that would allow a validator list publisher to attest
that a particular validator was associated with a domain. The idea may be
worth revisiting as a way of verifying the domain name claimed by the
validator's operator.
Resource limits were not properly applied to connections with
known IP addresses but no corresponding users.
Add unit tests for unlimited vs. limited ports.
An audit showed that a number of the RPC error codes in
ErrorCodes.h are no longer used in the code base. The unused
codes were removed from the file along with their support code
in ErrorCodes.cpp.
The ledger already declared a transaction that is both single-
and multi-signing malformed. This just adds some checking in
the signing RPC commands (like submit and sign_for) which allows
that sort of error to be identified a bit closer to the user.
In the process of adding this code a bug was found in the
RPCCall unit test. That bug is fixed as well.
If the number of peers a server has is below the configured
minimum peer limit, this commit will properly transition the
server's state to "disconnected".
The default limit for the minimum number of peers required was
0 meaning that a server that was connected but lost all its
peers would never transition to disconnected, since it could
never drop below zero peers.
This commit redefines the default minimum number of peers to 1
and produces a warning if the server is configured in a way
that will prevent it from ever achieving sufficient connectivity.
If a server is configured to support crawl, it will report the
IP addresses of all peers it is connected to, unless those peers
have explicitly opted out by setting the `peer_private` option
in their config file.
This commit makes servers that are configured as validators
opt out of crawling.
Several commands allow a user to retrieve a server's status. Commands
will typically limit disclosure of information that can reveal that a
particular server is a validator to connections that are not verified
to make it more difficult to determine validators via fingerprinting.
Prior to this commit, servers configured to operate as validators
would, instead of simply reporting their server state as 'full',
augment their state information to indicate whether they are
'proposing' or 'validating'.
Servers will only provide this enhanced state information for
connections that have elevated privileges.
Acknowledgements:
Ripple thanks Markus Teufelberger for responsibly disclosing this issue.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to responsibly
disclose any issues that they may find. For more on Ripple's Bug Bounty
program, please visit: https://ripple.com/bug-bounty
The /crawl API endpoint allows developers to examine the structure of
the XRP Ledger's overlay network.
This commit adds additional information about the local server to the
/crawl endpoint, making it possible for developers to create data-rich
network-wide status dashboards.
Related:
- https://developers.ripple.com/peer-protocol.html
- https://github.com/ripple/rippled-network-crawler
When deserializing specially crafted data, the code would ignore certain
types of errors. Reserializing objects created from such data results in
failures or generates a different serialization, which is not ideal.
Also addresses: RIPD-1677, RIPD-1682, RIPD-1686 and RIPD-1689.
Acknowledgements:
Ripple thanks Guido Vranken for responsibly disclosing these issues.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to responsibly
disclose any issues that they may find. For more on Ripple's Bug Bounty
program, please visit: https://ripple.com/bug-bounty
Specially crafted messages could cause the server to buffer large
amounts of memory which could increase memory pressure.
This commit changes how messages are buffered and imposes a limit
on the amount of data that the server is willing to buffer.
Acknowledgements:
Aaron Hook for responsibly disclosing this issue.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to
responsibly disclose any issues they may find. For information
on Ripple's Bug Bounty program, please visit:
https://ripple.com/bug-bounty
The constructor would previously assert that the specified buffer pointer
was non-null, even if the buffer size is specified as 0. While reasonable,
this also makes it more difficult to use this API.
* Using txnsExpected_, which is influenced by both the config
and network behavior, can reserve far too much or far too
little memory, wasting time and resources.
* Not an issue during normal operation, but a user could
cause problems on their local node with extreme configuration
settings.
* initFee was using a lot of logic that could look unclear. Add
some documentation explaining why certain values were used.
* Because initFee had side effects, callers needed to repeat the
max queue size computation, making the initial problem more
likely. Instead, return the max queue size value, so the caller
can reuse it.
* A newer test (testInFlightBalance()) was incorrectly using a
hard-coded queue limit. Fix it to use initFee's new return
value.
The --rpc_port command-line option is effectively ignored. We construct
an `Endpoint` with the given port, but then drop it on the floor.
(Perhaps the author thought the `Endpoint::at_port` method is a mutation
instead of a transformation.) This small change adds the missing
assignment to hold on to the new endpoint.
Fixes#2764
This changeset ensures the preferred ledger calculation
properly distinguishes the absence of trusted validations
from a preferred ledger which is the genesis ledger.
The `STObject` member function `setType()` has been renamed to
applyTemplate() and modified to throw if there is a template
mismatch.
The error description in the exception is, in certain cases,
used, to better indicate why a particular transaction was
considered ill formed.
Fixes#2585.
`Json::Value::isConvertibleTo` indicates that unsigned integers and
reals are convertible to string, but trying to do so (with
`Json::Value::asString`) throws an exception because its internal switch
is missing these cases. This change fills them in (and adds tests).
Acknowledgements:
Ripple thanks Guido Vranken for responsibly disclosing this issue.
Closes#2778
Although `parseURL` used a regex to pull the authority out of the URL
being parsed, it performed manual parsing of the hostname and port.
This commit rolls the parsing of the username and password, if any,
directly into the regex. The hostname can be a name, an IPv4 or an
IPv6 address.
Fixes#2751
* Adds local file:// URL support to the [validator_list_sites] stanza.
The file:// URL must not contain a hostname. Allows a rippled node
operator to "sideload" a new list if their node is unable to reach
a validator list's web site before an old list expires. Lists
loaded from a file will be validated in the same way a downloaded
list is validated.
* Generalize file/dir "guards" from Config test so they can be reused
in other tests.
* Check for error when reading validators.txt. Saves some parsing and
checking of an empty string, and will give a more meaningful error.
* Completes RIPD-1674.
* Relevant when deciding whether an account can queue multiple
transactions. If the potential spend of the already queued
transactions would dip into the reserve, the reserve is
preserved for fees.
* Also change several direct modifications of the owner count to
call adjustOwnerCount to preserve overflow checking.
* Update related unit testcase
* Resolves#2251
Perform some extra checks on the close time and sequence number
of a candidate for network consensus ledger. This tightens
defenses against some "insane/hostile supermajority" attacks.
Under certain conditions, we could call `memcpy` or `memcmp` with a null
source pointer. Even when specifying 0 as the amount of data to copy this
could result in undefined behavior under the C and C++ standards.
Acknowledgements:
Ripple thanks Guido Vranken for responsibly disclosing these issues.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to responsibly
disclose any issues that they may find. For more on Ripple's Bug Bounty
program, please visit: https://ripple.com/bug-bounty
The XRP Ledger is designed to be censorship resistant. Any attempt to
censor transactions would require coordinated action by a majority of
the system's validators.
Importantly, the design of the system is such that such an attempt is
detectable and can be easily proven since every validators must sign
the validations it publishes.
This commit adds an automated censorship detector. While the server is
in sync, the detector tracks all transactions that, in the view of the
server, should have been included and issues warnings of increasing
severity for any transactions which, have not after several rounds.
When Ed25519 support was added to ripple-lib, a way to specify
whether a seed should be used to derive a "classic" secp256k1
keypair or a "new" Ed25519 keypair was needed, and the
requirements were that:
1. previously seeds would, correctly, generate a secp256k1
keypair.
2. users would not have to know about whether the seed was
used to generate a secp256k1 or an Ed25519 keypair.
To address these requirements, the decision was made to encode
the type of key within the seed and a custom encoding was
designed.
The encoding uses a token type of 1 and prefixes the actual
seed with a 2 byte header, selected to ensure that all such
keypairs will, when encoded, begin with the string "sEd".
This custom encoding is non-standard and was not previously
documented; as a result, it is not widely supported and other
sofware may treat such keys as invalid. This can make it
difficult for users that have stored such a seed to use
wallets or other tooling that is not based on ripple-lib.
This commit adds support to rippled for automatically
detecting and properly handling such seeds.
The 'validation_seed' RPC command was used to change the validation
key used by a validator at runtime.
Its implementation was commented out with commit fa796a2eb5
which has been included in the codebase since the 0.30.0 release
and there are no plans to reintroduce the functionality at this
point.
Validator operators should migrate to using validator manifests
instead.
This fixes#2748.
The FeeEscalation amendment has been enabled on the XRP Ledger network
since May 19, 2016. The transaction which activated this amendment is:
5B1F1E8E791A9C243DD728680F108FEF1F28F21BA3B202B8F66E7833CA71D3C3.
This change removes all conditional code based around the FeeEscalation
amendment, but leaves the amendment definition itself since removing the
definition would cause nodes to think an unknown amendment was activate
causing them to become amendment blocked.
The commit also removes the redundant precomputed hashes from the
supportedAmendments vector.
Problem:
- There are only a few call sites to cachedRead, and all of them
currently do more work than is required since we know the type in each
case.
Solution:
- "Inline" the codepath to cachedRead, but do not check if the type is
valid. In all such call sites, we know the keylet to read directly.
This fixes#2550
The WaitableEvent class was a leftover from the pre-Boost
version of Beast and used Windows- and pthread-specific
APIs.
This refactor replaces that functionality by using only
interfaces provided by the C++ standard, making the code
more portable.
Closes#2402.
Many of the warnings on Windows were not resolved, just
silenced with _SILENCE_ALL_CXX17_DEPRECATION_WARNINGS.
They need to be resolved in a future commit.
* If rippled is not synced to the network, `fee` will return a
"no network" error instead of the possibly confusing "not enabled"
error.
* Resolves RIPD-1588
A validator that was configured to use a published validator list could
exhibit aberrent behavior if that validator list expired.
This commit introduces additional logic that makes validators operating
with an expired validator list bow out of the consensus process instead
of continuing to publish validations. Normal operation will resume once
a non-expired validator list becomes available.
This commit also enhances status reporting when using the `server_info`
and `validators` commands. Before, only the expiration time of the list
would be returned; now, its current status is also reported in a format
that is clearer.
A validator that was configured to use a published validator list could
exhibit aberrent behavior if that validator list expired.
This commit introduces additional logic that makes validators operating
with an expired validator list bow out of the consensus process instead
of continuing to publish validations. Normal operation will resume once
a non-expired validator list becomes available.
This commit also enhances status reporting when using the `server_info`
and `validators` commands. Before, only the expiration time of the list
would be returned; now, its current status is also reported in a format
that is clearer.
Problem:
- There are several specific overloads with some custom code that can be
easily replaced using Boost.Hex.
Solution:
- Introduce `strHex(itr, itr)` to return a string given a begin and end
iterator.
- Remove `strHex(itr, size)` in favor of the `strHex(T)` where T is
something that has a `begin()` member function. This allows us to
remove the strHex overloads for `std::string`, Blob, and Slice.
* For example Visual Studio, XCode. This will allow easily working with
any file in the IDE.
* Also ignore the file created by Visual Studio when using cmake
integration.
* Use conditional for unity/nounity sources (h/t @mellery451)
- allow private token for jenkins/codecov
- add custom targets for gcc/clang to generate codecov reports
- use CMake coverage target in jenkins build
- optional coverage_test argument when configuring the build
Reduces the account reserve for a multisigning SignerList from
(conditionally) 3 to 10 OwnerCounts to (unconditionally) 1
OwnerCount. Includes a transition process.
* When increasing the expected ledger size, add on an extra 20%.
* When decreasing the expected ledger size, take the minimum of the
validated ledger size or the old expected size, and subract another 50%.
* Update fee escalation documentation.
* Refactor the FeeMetrics object to use values from Setup
As described in #2314, when an offer executed with `Fill or Kill`
semantics, the server would return `tesSUCCESS` even if the order
couldn't be filled and was aborted. This would require additional
processing of metadata by users to determine the effects of the
transaction.
This commit introduces the `fix1578` amendment which, if enabled,
will cause the server to return the new `tecKILLED` error code
instead of `tesSUCCESS` for `Fill or Kill` orders that could not
be filled.
Additionally, the `fix1578` amendment will prevent the setting of
the `No Ripple` flag on trust lines with negative balance; trying
to set the flag on such a trust line will fail with the new error
code `tecNEGATIVE_BALANCE`.
Fixes: RIPD-1648
- use ExternalProject for snappy, lz4, SOCI, and sqlite3
- use FetchContent for NuDB
- update SOCI from 79e222e3c2278e6108137a2d26d3689418b37544 to
3a1f602b3021b925d38828e3ff95f9e7f8887ff7
- update lz4 from c10863b98e1503af90616ae99725ecd120265dfb to v1.8.2
- update sqlite3 from 3.21 to 3.24
- update snappy from b02bfa754ebf27921d8da3bd2517eab445b84ff9 to 1.1.7
- update NuDB from 00adc6a4f16679a376f40c967f77dfa544c179c1 to 1.0.0
When validators publish a proposal, they include the close time that they
believe the new ledger should have, and the network attempts to reach
consensus on that.
Instead of delaying consensus if no close time has the required majority
the servers can "agree to disagree"; if this happens, they switch to
proposing a close time of 0, and the network avalanches to that value.
If that occurs, determinstic rules record the new ledger's close time as
being one second later than its parent, and set a flag indicating that
no consensus on the close time was reached.
The wire protocol decoder would incorrectly filter such proposals, so
that they would not be seen by the higher level consensus engine.
This commit removes the low-level filtering, and allows higher level
code to filter out stale proposals instead.
In order to facilitate transaction signing, `rippled` offers the `sign` and
`sign_for` and `submit` commands, which, given a seed, can be used to sign or
sign-and-submit transactions. These commands are accessible from the command
line, as well as over the WebSocket and RPC interfaces that `rippled` can be
configured to provide.
These commands, unfortunately, have significant security implications:
1. They require divulging an account's seed (commonly known as a "secret
key") to the server.
2. When executing these commands against remote servers, the seeds can be
transported over clear-text links.
3. When executing these commands over the command line, the account
seed may be visible using common tools that show running processes
and may potentially be inadvertently stored by system monitoring
tools or facilities designed to maintain a history of previously
typed commands.
While this commit cannot prevent users from issuing these commands to a
server, whether locally or remotely, it restricts the `sign` and `sign_for`
commands, as well as the `submit` command when used to sign-and-submit,
so that they require administrative privileges on the server.
Server operators that want to allow unrestricted signing can do so by
adding the following stanza to their configuration file:
[signing_support]
true
Ripple discourages server operators from doing so and advises against using
these commands, which will be removed in a future release. If you rely on
these commands for signing, please migrate to a standalone signing solution
as soon as possible. One option is to use `ripple-lib`; documentation is
available at https://developers.ripple.com/rippleapi-reference.html#sign.
If the commands are administratively enabled, the server includes a warning
on startup and adds a new field in the resulting JSON, informing the caller
that the commands are deprecated and may become unavailable at any time.
Acknowledgements:
Jesper Wallin for reporting this issue to Ripple.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to responsibly
disclose any issues that they may find. For more on Ripple's Bug Bounty
program, please visit: https://ripple.com/bug-bounty
Exclude several libraries from build when we are included in a
super-project (this is the case when someone only wants to use
xrpl_core). Force several target (deprecated) params to be cache
variables since they are now exposed as options.
Increase page size for SQLite transaction database upon creation
Provide diagnostics for transaction db page usage.
Shut down rippled gracefullly if transaction db is running out of pages.
Add new rippled maintenance command line option to cause new page size
to take effect.
Switch to target-oriented dependencies. Use imported targets for
dependencies (openssl, boost). Localize FindBoost to remove cmake
version dependence for latest boost support. Logically separate
"ripple-libpp" core sources and add install targets.
Add ninja build for msvc. Add two clang sanitizer builds. Misc script
changes to work with latest modernized cmake.
Certain versions of the Beast HTTP & WebSocket library can
generate exceptions, which unless caught, will result in
unexpected behavior.
Acknowledgements:
Ripple thanks Thomas Snider for originally noticing this
issue and responsibly disclosing it to Ripple.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers
to responsibly disclose any issues that they may find. For
more on Ripple's Bug Bounty program, please visit:
https://ripple.com/bug-bounty
Based on a TODO comment in DisputedTX.h, it seems at one point the
data members of ConsensusParms were macros. Now that they are not,
we should spell them like other data members (without all uppercase).
After some discussion on https://github.com/ripple/rippled/pull/2595
we have decided that the allocator should not be plumbed through
the KeyCache class template. As such, remove the comment suggesting
to push the allocator through.
All listed validators are trusted and quorum is 80% of trusted
validators regardless of the number of:
* configured published lists
* listed or trusted validators
* recently seen validators
Exceptions:
* A listed validator whose master key has been revoked is not trusted
* Custom minimum quorum (specified with --quorum in the command line)
is used if the normal quorum appears unreachable based on the number
of recently received validators.
RIPD-1640
std::random_shuffle is deprecated in C++14 and removed completely
in C++17. The two-iterator version of std::random_shuffle usually
depends on std::rand and also on a global state. The preferred
replacement is to use std::shuffle with a pseudo-random number
generator.
- Since we require a min Boost version of 1.67 as of recently (for
Beast), we also remove the conditional checks that existed for us
to know whether Boost.Process is available or not. We can
always assume it is available now.
- Remove runtime checks for minimum Boost and OpenSSL versions
since they are checked at CMake configure time.
This changes the rules for payments in two ways:
1) It sets the maximum number of offers any book step can consume from
2000 to 1000.
2) When a strand contains a step that consumes too many offers,
currently the liquidity is not used at all and the strand will
be considered dry. This changes things so the liquidity is used,
however the strand will still be considered dry.
* mFeeDue is only used in one place by one derived class, so
only compute it as a local in that function.
* The baseFee needs to be calculated outside of the Transactor class
because, it can change during transaction processing, and the function
is static, so we need to be sure to call the right version
* Rename Transactor::calculateFee to minimumFee
Fixes: RIPD-1574
Alias beast address classes to the asio equivalents. Adjust users of
address classes accordingly. Fix resolver class so that it can support
ipv6 addresses. Make unit tests use ipv6 localhost network. Extend
endpoint peer message to support string endpoint
representations while also supporting the existing fields (both are
optional/repeated types). Expand test for Livecache and Endpoint.
Workaround some false positive ipaddr tests on windows (asio bug?)
Replaced usage of address::from_string(deprecated) with free function
make_address. Identified a remaining use of v4 address type and
replaced with the more appropriate IPEndpoint type (rpc_ip cmdline
option). Add CLI flag for using ipv4 with unit tests.
Release Notes
-------------
The optional rpc_port command line flag is deprecated. The rpc_ip
parameter now works as documented and accepts ip and port combined.
Under some corner cases, the base64 decoder would not allocate
enough memory, which could result in spurious errors.
Acknowledgements:
Ripple thanks Guido Vranken for originally noticing this issue
and responsibly disclosing it to Ripple.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers
to responsibly disclose any issues that they may find. For
more on Ripple's Bug Bounty program, please visit:
https://ripple.com/bug-bounty
Fixes: RIPD-1575. Fix argument passing to runner. Allow multiple unit
test selectors to be passed via --unittest argument. Add optional
integer priority value to test suite list. Fix several failing manual
tests. Update CLI usage message to make it clearer.
The lsfDepositAuth flag limits the AccountIDs that can deposit into
the account that has the flag set. The original design only
allowed deposits to complete if the account with the flag set also
signed the transaction that caused the deposit.
The DepositPreauth ledger type allows an account with the
lsfDepositAuth flag set to preauthorize additional accounts.
This preauthorization allows them to sign deposits as well. An
account can add DepositPreauth objects to the ledger (and remove
them as well) using the DepositPreauth transaction.
Add a new invariant checker that verifies that we never charge a
fee higher than specified in the transaction; we will charge less
in some corner cases where the transacting account cannot afford
the fee.
Detect more anomalous conditions, and improve the logged error
messages.
Clarify the code flow associated with invoking the invariant checker
from `Transactor`, add extra comments and improve naming to make the
code self-documenting.
The six different ranges of TER codes are broken up into six
different enumerations. A template class allows subsets of
these enumerations to be aggregated. This technique allows
verification at compile time that no TEC codes are returned
before the signature is checked.
Conversion between TER instance and integer is provided by
named functions. This makes accidental conversion almost
impossible and makes type abuse easier to spot in the code
base.
* UptimeClock is a chrono-compatible seconds-precision clock.
* Like UptimeTimer, its purpose is to make it possible for clients
to query the uptime thousands of times per second without a
significant performance hit.
* UptimeClock decouples itself from LoadManager by managing its
own once-per-second update loop.
* Clients now traffic in chrono time_points and durations instead
of int.
Some classes had virtual methods, but were missing a virtual
destructor.
Technically, every unit test that inherits from the Beast test suite
would get flagged by `-Wnon-virtual-dtor` but I did not think it would
be a great idea to go sprinkle a virtual destructor for every Ripple
test suite.
* Tally and duration counters for Job Queue tasks and RPC calls
optionally rendered by server_info and server_state, and
optionally printed to a distinct log file.
- Tally each Job Queue task as it is queued, starts, and
finishes running. Track total duration queued and running.
- Tally each RPC call as it starts and either finishes
successfully or throws an exception. Track total running
duration for each.
* Track currently executing Job Queue tasks and RPC methods
along with durations.
* Json-formatted performance log file written by a dedicated
thread, for above-described data.
* New optional parameter, "counters", for server_info and
server_state. If set, render Job Queue and RPC call counters
as well as currently executing tasks.
* New configuration section, "[perf]", to optionally control
performance logging to a file.
* Support optional sub-second periods when rendering human-readable
time points.
Each validator will generate a random cookie on startup that it will
include in each of its validations. This will allow validators to detect
when more than one validator is accidentally operating with the same
validation keys.
* The compiler can provide many non-explicit constructors for
aggregate types. This is sometimes desired, but it can
happen accidentally, resulting in run-time errors.
* This commit assures that no types are aggregates unless existing
code is using aggregate initialization.
We had several hash functions implemented, including SipHash,
SpookyHash and FNV1a.
Default to using xxhash and remove the code for the remaining
hash functions.
For the functions defined in <ctype.h> the C standard requires
that the value of the int argument be in the range of an
unsigned char, or be EOF. Violation of this requirement
results in undefined behavior.
When creating an escrow, if the `CancelAfter` time is specified but
the `FinishAfter` is not, the resulting escrow can be immediately
completed using `EscrowFinish`. While this behavior is documented,
it is unintuitive and can be confusing for users.
This commit introduces a new fix amendment (fix1571) which prevents
the creation of new Escrow entries that can be finished immediately
and without any requirements.
Once the amendment is activated, creating a new Escrow will require
specifying the `FinishAfter` time explicitly or requires that a
cryptocondition be specified.
Constructing deeply nested objects could allow an attacker to
cause a server to overflow its available stack.
We now enforce a 10-deep nesting limit, and signal an error
if we encounter objects that are nested deeper.
Acknowledgements:
Ripple thanks Guido Vranken for responsibly disclosing this
issues.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled codebase and urge reviewers
to responsibly disclose any issues that they may find. For
more on Ripple's Bug Bounty program, please visit
https://ripple.com/bug-bounty
Constructing deeply nested objects could allow an attacker to
cause a server to overflow its available stack.
We now enforce a 10-deep nesting limit, and signal an error
if we encounter objects that are nested deeper.
Acknowledgements:
Ripple thanks Guido Vranken for responsibly disclosing this
issues.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled codebase and urge reviewers
to responsibly disclose any issues that they may find. For
more on Ripple's Bug Bounty program, please visit
https://ripple.com/bug-bounty
* RIPD-1617, RIPD-1619, RIPD-1621:
Verify serialized public keys more strictly before
using them.
* RIPD-1618:
* Simplify the base58 decoder logic.
* Reduce the complexity of the base58 encoder and
eliminate a potential out-of-bounds memory access.
* Improve type safety by using an `enum class` to
enforce strict type checking for token types.
* RIPD-1616:
Avoid calling `memcpy` with a null pointer even if the
size is specified as zero, since it results in undefined
behavior.
Acknowledgements:
Ripple thanks Guido Vranken for responsibly disclosing these
issues.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers
to responsibly disclose any issues that they may find. For
more on Ripple's Bug Bounty program, please visit:
https://ripple.com/bug-bounty
The function is non-virtual and hides the virtual function specified in
the base class.
Falling back to the virtual function in the base class is the correct
solution.
Fixes: RIPD-1601
Fix intermittent failure in server stream sub/unsub test.
Root cause is LoadManager thread *sometimes* running and causing a
fee change event which got published before our test could
unsubscribe. Fixed by explicitly stopping the LoadManager for this test.
These changes use the hash of the consensus transaction set when
characterizing the mismatch between a locally built ledger and fully
validated network ledger. This allows detection of non-determinism in
transaction process, in which consensus succeeded, but a node somehow
generated a different subsequent ledger.
When a test suite starts and ends, it informs the parent process. If the parent
has received a start message without a matching end message it reports that a
child may have crashed in that suite.
Fixes: RIPD-1521
Switch to pure doxygen HTML for developer docs. Remove docca/boostbook
system. Convert consensus document to markdown. Add existing markdown
files to doxygen input set. Fix some image paths and scale images for
use with MD links. Rename/cleanup some files for consistency.
Add pipeline logic for windows slaves. Add ninja and parallel test run
option. Add make doc target build in build-and-test.sh. Cleanup README
files. Add nounity windows build. Add link to jenkins summary table.
Add rippled_classic build (win). Improve formatting of summary table.
* Rename isArray to isArrayOrNull
* Rename isObject to isObjectOrNull
* Introduce isArray and isObject
* Change as many uses of isArrayorNull to isArray as possible
* Change as many uses of isObjectorNull to isObject as possible
* Reject null JSON arrays for subscribe and unsubscribe
* Remove all builds except cmake gcc & clang debug.
* Time some dependency and build operations, using a custom format to avoid
interfering with other timers.
* Use Travis's "trusty" infrastructure.
* Install more dependencies via apt.
* Update boost version to 1.65.1.
* Do not run unit tests under gdb - several security features prevent
it from running correctly.
* Run test job with two processes.
* Replace Ubuntu scons dependency with cmake
* Remove dependency installation support for Ubuntu 12.04 Precise Pangolin.
It is EOL'd as of 2017-04-28. http://releases.ubuntu.com/12.04/
The assert in LedgerHistory.cpp was incorrect since
getLedgerByHash() can return a null object. The assert has
been seen to fire inappropriately during online_delete.
The assert in LedgerMaster.cpp was simply misguided. If a
null hash were returned by getLedgerHashForHistory(), then we
should not forward that hash to peers. The assert only fires
in a debug build, so a release build would forward the request
for a null hash to peers.
These changes augment the Validations class with a LedgerTrie to better
track the history of support for validated ledgers. This improves the
selection of the preferred working ledger for consensus. The Validations
class now tracks both full and partial validations. Partial validations
are only used to determine the working ledger; full validations are
required for any quorum related function. Validators are also now
explicitly restricted to sending validations with increasing ledger
sequence number.
Per issue #2354, when the log level of a server was configured at
"trace", sensitive keying meterial generated by the `wallet_propose`
command could be written to the server's log file, if one was
configured.
This commit improves the log scrubbing code to account for the
sensitive information generated by a `wallet_propose`.
** Important security consideration **
We still caution everyone *against* executing this command on a
server that they do not control: a malicious server operator could
intercept the generated keypair, or operate a modified server that
returns keypairs that are not securely generated.
* This change passes detailed error messages from the JSON parser
on the server side, back to the client for inclusion into the
reply's error message.
* Errors originating from the server's inability to parse are
reclassified from rpcINTERNAL to rpcINVALID_PARAMS.
Introduce a new ledger type: ltCHECK
Introduce three new transactions that operate on checks:
- "CheckCreate" which adds the check entry to the ledger. The
check is a promise from the source of the check that the
destination of the check may cash the check and receive up to
the SendMax specified on the check. The check may have an
expiration, after which the check may no longer be cashed.
- "CheckCash" is a request by the destination of the check to
transfer a requested amount of funds, up to the check's SendMax,
from the source to the destination. The destination may receive
less than the SendMax due to transfer fees.
When cashing a check, the destination specifies the smallest
amount of funds that will be acceptable. If the transfer
completes and delivers the requested amount, then the check is
considered cashed and removed from the ledger. If enough funds
cannot be delivered, then the transaction fails and the check
remains in the ledger.
Attempting to cash the check after its expiration will fail.
- "CheckCancel" removes the check from the ledger without
transferring funds. Either the check's source or destination
can cancel the check at any time. After a check has expired,
any account can cancel the check.
Facilities related to checks are on the "Checks" amendment.
Do not process a transaction received from a peer if it has
been processed within the past ten seconds.
Increase the number of transaction handlers that can be in
flight in the job queue and decrease the relative cost for
peers to share transaction and ledger data.
Additionally, make better use of resources by adjusting the
number of threads we initialize, by reverting commit
68b8ffdb63.
Performance counter modifications:
* Create and display counters to track:
1) Pending transaction limit overruns.
2) Total peer disconnections.
3) Peers disconnections due to resource consumption.
Avoid a potential double-free in Json library.
In some cases this script did not detect properly that the subprocess is already terminated. The "append" loop afterwards caused memory to fill up and the script to fail eventually.
The DepositAuth feature allows an account to require that
it signs for any funds that are deposited to the account.
For the time being this limits the account to accepting
only XRP, although there are plans to allow IOU payments
in the future.
The lsfDepositAuth protections are not extended to offers.
If an account creates an offer it is in effect saying, “I
will accept funds from anyone who takes this offer.”
Therefore, the typical user of the lsfDepositAuth flag
will choose never to create any offers. But they can if
they so choose.
The DepositAuth feature leaves a small gap in its
protections. An XRP payment is allowed to a destination
account with the lsfDepositAuth flag set if:
- The Destination XRP balance is less than or equal to
the base reserve and
- The value of the XRP Payment is less than or equal to
the base reserve.
This exception is intended to make it impossible for an
account to wedge itself by spending all of its XRP on fees
and leave itself unable to pay the fee to get more XRP.
This commit
- adds featureDepositAuth,
- adds the lsfDepositAuth flag,
- adds support for lsfDepositAuth in SetAccount.cpp
- adds support in Payment.cpp for rejecting payments that
don't meet the lsfDepositAuth requirements,
- adds unit tests for Payment transactions to an an account
with lsfDepositAuth set.
- adds Escrow and PayChan support for lsfDepositAuth along
with as unit tests.
* Null json values can be objects or arrays.
* json arrays are now interpreted as batch commands.
* json objects are single commands.
* null jsons are ambiguous as to whether they are single or batch
commands and should be avoided.
In connection with Ripple's decentralization strategy, the example
validators.txt file is being updated, and the [validators] field,
which contained a static list of trusted validators, is being
replaced with two new fields:
- The [validator_list_keys] field which contains the keys that can be
used to sign its recommended validator list.
- The [validator_list_sites] field which specifies the URLs where
dynamic validator list can be found.
The initial configuration links to Ripple's default validator list,
which is located at https://vl.ripple.com.
To read more about Ripple's decentralization strategy, please visit
the Ripple Dev Blog:
https://ripple.com/dev-blog/decentralization-strategy-update/
Profiling and research indicates that the SQLite query planner executed
our existing SQL queries sub-optimally by not using the index efficiently.
Restructuring the SQL query works around this issue and allows queries
to be executed efficiently and without unnecessary delay.
c10863b98 fix : asan error in dctx, due to increased maximum frame header size, reported by Craig Young
1e92bb0af Merge pull request #382 from lz4/installVars
edb434365 updated lz4frame manual
a1f3a27e9 Merge branch 'dev' into installVars and fixed conflicts
af9d72b7f Merge pull request #383 from lz4/blockChecksum
8593ba883 Merge pull request #380 from lz4/dictID
01cdbfb5f lz4c legacy commands are now enabled at runtime based on link/binary name "lz4c"
731cff120 fix minor markdown display issues
aea7d521a better respect GNU standard Makefile conventions
930a69211 cli : restored command -BX to enable block checksum (#322)
77f99d292 restored block checksum capability at lz4frame API level
228e26ee6 Merge pull request #378 from deymo/dev
757497ae3 implemented lz4frame decompression API
4531637ec support dictionary compression with independent blocks
8d597d62d fixed gcc prototype warning
d8aafe2c5 dictionary compression correctly uses compression level
ca2fb166a fixed C++ conversion warnings
bf8daa2fd fixed uninitialization error in lz4frame
31f2cdf4d implemented dictionary compression in lz4frame
1d1737aaf fixed frameCompress example
a82dadfba added dictID inside LZ4F_frameInfo_t
e98a52857 updated Frame specification
76ef6d0ab Allow to predefine FORCE_INLINE macro.
9a967030d Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
c5a015bc3 Merge pull request #376 from ido/patch-1
4891f909e Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
dba4d70a1 updated man page
8aeaf6bb3 Fix typos preventing installation of static lib.
ac5ed8e4b Merge pull request #374 from ferdnyc/patch-2
63d500568 Generate updated man page from Markdown source
ef0655510 Merge pull request #373 from ferdnyc/patch-2
872f767fe Fix formatting of concatenation example
6ad3a983d fix#369
2ef4afeeb Merge pull request #371 from jku/tests-LIBDIR
e14b4c5a3 tests/Makefile: don't use LIBDIR as variable
1525fd1f5 Merge pull request #367 from Chocobo1/fallthrough
7982a905e Merge pull request #368 from Chocobo1/readme
9d453a188 Update README.md
a4ec7fa7b Fix gcc7 Wimplicit-fallthrough warnings
89b9f026c made level 10 a bit faster
a6fd0f9d0 -g compilation flag not by default for lz4 cli
bfc180660 clarified lz4frame api comment (#350)
d18084c65 report where decompression ends (#313)
139a387d1 updated NEWS
0beaa356f fixed minor scan-build warning
775e63ee0 refactored simple_buffer.c example (#363)
7e15e240a added a paragraph on overlap matches
03d8586fc cli accept block sizes with KB / MB prefixes
8c355d17f Merge pull request #360 from Chocobo1/md
1c9d4091d [Doc] Fix markdown
6614be7cc Merge pull request #359 from PierreNav/dev
4e359f5c3 Add DLL files to the INSTALL target
5c97cdfa0 Merge pull request #352 from lz4/resetDCtx
2600a154b fix (minor) g++ compatibility for frametest
e60cbb5ca added test for LZ4F_resetDecompressionContext()
2012e4de9 fixed c_standards tests
a90212761 bumped version number to 1.8.0
b8575f2d2 updated Makefile
fe932c452 expose LZ4F_resetDecompressionContext()
1efa48831 minor readability changes
a8dd86d93 changed macro HEAPMODE into LZ4_HEAPMODE
11bfedb6c Merge pull request #349 from svpv/master
6cf1f7d75 lz4hc.c: clamp compression levels > 12
a41df535f lz4cli.c: fix a comment: LZ4HC_DEFAULT_CLEVEL -> LZ4HC_CLEVEL_MAX
f3460fc14 liz4hc.h: fix a comment: LZ4HC_MAX_CLEVEL -> LZ4HC_CLEVEL_MAX
37ef330cd Merge pull request #348 from terrelln/deprecate
0b3e80704 [LZ4F] Allow users to disable LZ4F_DEPRECATE
31e9ed612 Merge pull request #347 from lz4/negativeCLevels
e2c9b1912 lz4frame : Added negative compression levels
7eecd32c0 ensure lz4f_cctx internal buffer size remain valid in case of malloc error
e169edac0 lz4frame : control lz4 context creation success
e2827775e make __packed memory access default for gcc
3d4ee35da Merge pull request #344 from lz4/LZ4F_getFrameInfo
6226d52ea fixed minor Visual warning
b88df6b1b Improved comments on LZ4F_getFrameInfo()
fc31257ab added LZ4F_resetDecompressionContext()
f0a7651fc Safer LZ4_getFrameInfo()
ab547a0ef Merge pull request #342 from iburinoc/isatty
9b1f00056 Fix inline compile errors
afde27acf Fix IS_CONSOLE returning 1 for NUL on windows
84246b974 Merge pull request #341 from iburinoc/exematch
4567d0def Merge pull request #340 from lz4/optlz4opt
e9c3b14f2 Ignore extensions in exe name matching
38362c1d6 Merge pull request #338 from ldv-alt/dev
60b6d2907 improved level 10 speed for degenerated cases
8f8fc52fc Merge branch 'dev' into optlz4opt
0863931a1 fixed API comment for LZ4F_createCompressionContext()
dab3590fc Export deprecated symbols declared in lz4hc.h
13f5cb2d5 Merge pull request #337 from ldv-alt/dev
b1daffc4e cli: add GNU separator -- specifying that all following arguments are files
68044acfe Merge branch 'optlz4opt' of github.com:Cyan4973/lz4 into optlz4opt
b8bc70022 minor refactor
cea8f60df slight btopt speed improvement
f513020a6 slight btopt speed improvement
8bd32a17b made SET_PRICE macro more usable
118b47f3d improved lz4opt speed (~4%)
eff6166eb minor price function optimization
0d073d4d2 added `extern C` for lz4.h static section
aae447fff LZ4_compress_HC_continue_destSize() works as intended up to level 10
cd35f0d98 LZ4_compress_HC_destSize() uses LZ4HC_compress_generic() code path
6b2a1b360 updated lz4 version number
430b7d32b created LZ4_HC_STATIC_LINKING_ONLY section
9144520dc fixed Visual compilation error
a33bf89c6 fix#332 : do not modify /dev/null permissions
e945a27f9 fix#333 : expose obsolete decoding functions
e1b2b370e added Oleg @remittor contribution
b9132bedc Merge pull request #331 from ldv-alt/dev
50fe87f13 Merge pull request #329 from remittor/dev-hc2
883ebdcee Export only those symbols that are part of public API
25b243588 Export deprecated symbols
7aeecbff7 Explicitly set visibility of public API functions when gcc is used
93b3b08ef Merge pull request #330 from inikep/dev
cb1671955 bench.c: respect LZ4_MAX_INPUT_SIZE limit
517d446d7 Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
36842ebb1 lz4hc: Fix LZ4HC_compress_hashChain for backward compatibility
baa155088 lz4hc: Fix LZ4HC_compress_hashChain for full support destSize variant
66b26a389 tests: fuzzer: Add test for LZ4_compressHC_destSize
f007153e3 lz4hc: Add LZ4_compressHC_destSize and LZ4_compress_HC_continue_destSize
45b592b7e lz4hc: Cleanup function LZ4HC_encodeSequence
534f8fa5d lz4hc: Cleanup function LZ4HC_compress_hashChain
9141ada8d Merge pull request #328 from ldv-alt/dev
0dfb0b9da Fix test-lz4-basic
97df1c978 updated LICENSE
5bfdff96b Merge pull request #320 from inikep/dev
d3eaf7883 remove MOREFLAGS from test-lz4c32
5bb6ba794 Merge pull request #319 from inikep/dev
ca2c5fd3c CFLAGS=-I/usr/include/x86_64-linux-gnu
ace22424d circle.yml: fixed lz4c32 target (2)
0d77c85d9 circle.yml: fixed lz4c32 target
24007bb4e Merge pull request #318 from esiegerman/eks/mklib
4ee9bd5c3 improved Circle CI tests
c0a8d0ad8 added circle.yml
3ec1591db Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
68a358769 Avoid fseek()'s 2GiB barrier with MinGW
40ad1e85d upgraded util.h
eb51b2b8d Explicitly create $(DESTDIR)$(LIBDIR)/ at install time
69dbafc1c Merge pull request #317 from esiegerman/eks/no-andand-oror
b89cac7b2 Don't use "foo && false || true"
7e57c398c Merge pull request #316 from inikep/dev
f4b9c1d03 Merge pull request #315 from bebehei/patch-1
c139eb40b added OpenBSD NetBSD SunOS to list of supported `make install` OSes
5e1a925be update repolink in makefile
272699a4e Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
c7b14967a updated platform.h
4f261b828 Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
04e5eaf9b added DragonFly to list of supported `make install` OS
04ec09269 updated comments on block sizes
7ed257191 Merge pull request #312 from mgorny/cmake-pc-version
96e150e7d [cmake] Fix substituting version in liblz4.pc
67abd75e6 Merge pull request #307 from inikep/dev
8e1fd97d1 lz4frame.h: added Introduction
d56ee32b3 updated gen-lz4-manual.sh
f54c7e0e7 added lz4frame_manual.html
9071df0fa lz4frame.h: prepared to generate manual
6a5633e99 lz4 manual updated to v1.7.5
f9f48f8ed lz4.h: improved manual generation
128244371 improved gen_manual
489d1618c added "This Makefile is validated for"
339e79320 Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
ca083dd4d Merge pull request #304 from Chocobo1/cppcheck
db6f733ec Use logical or instead of bitwise or
44f95e92e Fix printf specifier
9683a1ae8 LZ4_MEMORY_USAGE can be modified from compilation command line
16d69c8d0 Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
05e27ade6 updated NEWS
c18a81c02 Merge pull request #296 from mgorny/cmake-fixes
d7969e49a cmake: Install manpages
1380c33b7 cmake: Install lz4cat and unlz4 symlinks
28db4acc9 cmake: Support building shared & static libs simultaneously
9e867db90 cmake: Fix SOVERSION to match Makefiles
cb9599449 Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
8c4de60d0 Merge pull request #295 from inikep/opt-parser
d2b51c22d fuzzer: tests more compression levels
312d88249 removed nextToUpdateBT
3d5bb3897 clean logging
1c80b9af4 LZ4HC_getSearchNum
581da2bf8 Merge pull request #293 from inikep/dev
98f9d6c72 improved logging
12aae846b fixed table update in LZ4_loadDictHC
c8b31263c added FUZ_CLEVEL_DEFAULT
7a73c5c1f changed default PREFIX and MANDIR
d57ff6456 changed default PREFIX and MANDIR
a3d61cf3d improved test-lz4-opt-parser
20381a2fc fixed -BD compression
f4575f4f1 added test-lz4-opt-parser
e6536faf3 lib\Makefile: fixed INSTALL_DATA
efaa9b7bc Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
b22e2bed5 BSD: improved "make install"
ea51ad0ce Solaris: working "make install"
19c0f21b0 updated Makefile : lz4 no longer recompiled when already up-to-date
9b4b081c1 "make test" is now compatible with Solaris
b41f9bb13 Merge pull request #292 from inikep/dev
7eb16d97d updated NEWS
52cac9a97 updated a few macros names
fea95c156 use FindFirstFileA and FindNextFileA on Windows
e6af952f8 improved comments
581245063 use fseeko for 32-bit MacOS
f084b6538 test Large File support for Mac OS-X in 32-bits mode
973bc7974 util.h and platform.h based on zstd
70526a11e fixed lib/clean
bc0839c5d minor fix for travis-install test
7cf0bb97b LZ4F_compressBound(0) provides upper bound for LZ4F_flush() and LZ4F_compressEnd() [#290, suggested by @vtermanis]
385cb4f53 minor update Makefile
4f61505cc datagen.vcxproj: CharacterSet is MultiByte
253ac12cb VS projects: CharacterSet changed to MultiByte
287f7d3d6 added IncludePath
2fd7eb554 fixed Visual Studio compilation
3c6ce9c66 fixed gcc 4.4 support
64cbc4e1d improved MinGW support
9546ba62d executables use basic types from util.h
452cc970d reordered definitons in platform.h
090cb887a improved formatting (2)
e1332241a improved formatting
5f0cc8e2d fixed gcc warnings (2)
2efedaf7d fixed gcc warnings
6adf05d1e improved platform.h
2cfef1395 executables use platform.h
bb69cdf25 updated util.h
45b14ab8e added platfom.h
913b98fd6 fix `make` concurrency build (#277)
fec57e49e Merge pull request #287 from inikep/dev
4f050b68d tree update dependent from compression level
838ed341f fullUpdate as a parameter
b3ecc3e3e last match starts at least 12 bytes before end of block
d694bf911 faster opt compression
e7c42d757 Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
a22e71d4a full binary tree update
626552671 updated NEWS
de93e9e5d fix#285 : lz4cat doesn't work with stdin (reported by @beiDei8z)
fb6c98c85 slightly improved lz4opt.h
20183ad7a Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
0280cf40d fixed path-1, as suggested by @t-mat
7e2d4c4c0 fixed test-lz4c32
750a50cf6 fix#284, reported by @beiDei8z : added test cases for path/to/lz4cat
01ffed25a fix : invocation of path/to/unlz4 (#284), reported by @beiDei8z
8c2f23c13 fixed LZ4_STREAMHCSIZE
76228f064 LZ4HC_DICTIONARY_LOGSIZE 17
8bfa42e61 improved ratio
128903824 15-bit LZ4HC_HASH_LOG
410ba53db improved LZ4HC_BinTree_GetAllMatches
1833be1cf fixed gcc warnings
f2ebf37bf slightly improved ratio
77b051ed7 improve code formatting
e3fee9474 LZ4HC_MAX_CLEVEL = 12
757ef1d5f removed LZ4HC_GetAllMatches
cfe6a24b0 updated LZ4HC_get_price
2113ead17 17-bit LZ4HC_MAXD
c1ef7a177 introduced LZ4HC_compress_hashChain
3f430daf7 added lz4opt.h
5ae0e789f Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
5eb547a60 fix#283 : implement LZ4_versionString().
9a6f7dfc5 updated NEWS
a9a8dbf7c fixed minor display error when using -r
6b90cfbea Updated man page
009f06e01 minor man page update
b3bc054d0 Merge pull request #279 from t-mat/md-to-man
544290d58 Add "man" target to Makefile
018ddf799 numbering
605512a7b updated NEWS
f4c74e6fa library release build compatible with environment variable
edc132b69 lz4 release build compatible with PIE
7819d5878 new test case with fPIE
de0cf5de6 fixed LZ4IO_compressMultipleFilenames
65a542579 updated README.md
2cbd0233b Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
cc21115cb minor warning fix
8875e7dbb fix minor analyzer warning
8b233b228 added a few dates
ba7a85b46 fixed minor analyzer warning
3bde02a05 added gcc 4.4 test
2fe3aa985 added examples to make all
742f2b683 added doc in code comments
2dbcebfda changes travis test order
b4db9d7e3 Add C++14 standard [[deprecated]] attribute
a38cbf9f6 fixed : shadow global variable on gcc 4.4 (minor)
19df3029e fixed#272 (compilation fails on gcc 4.4), reported by @totaam
519932afa attempt -mx32 tests on Travis CI
a91983daa highly improved speed on -mx32 mode Now -mx32 is fastest mode on x64 CPU
6cabcd151 removed usan32 tests. Let's summarize : gcc-5 and gcc-6 on Travis do not support UB sanitizer. And clang on Travis is not able to generate -m32 code. Great. It makes the combination impossible to test on Travis ... :(
7a22904a7 moved usan32 tests to gcc-6
ffc8c18c1 usan32 relies on default compiler (by default)
a4f4bf21e fixed gcc-5 conf
71814a26f switch usan32 tests to gcc-5
8e1ea59ad attempt to fix sanitize32 package dependency
8f536a43b added sanitize32 tests to Travis
3f6f57768 fixed strict aliasing warnings in fasttest
c0e55ec05 fixed versionsTest
7d5354847 added v1.x.y to versions tests
279344a17 fixed Travis test cases
1abecbc33 fix 32-bits mode. Large File support for Mac OS-X in 32-bits mode Fixed potential undefined behavior Changed makefile for 32-bits mode
7fde7438d Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
a22ca9378 updated README for v1.7.3
e9bec0fdc Merge pull request #271 from inikep/dev
b1519f861 Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
2dee7bdaf include NEWS in lz4_x64.zip and lz4_x86.zip
632ab3b8d updated programs\README.md
8b48d5e44 updated example\README.md
11f6217cc created dll\example directory
873206e70 appveyor.yml: create only lz4_x64.zip and lz4_x86.zip
7b6a9d863 Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
2e10aab20 MinGW doesn't require the import library at all
7cecc7b08 fixed a limit case scenario
00fa1b9c0 fixed double ;; for Visual
929badd61 fixed sign conversion warning
d613e2494 restored .travis.yml
d2be69b14 fixed minor coverity warnings
8c32a12f1 .travis.yml for coverity
2eddc8ea2 removed .travis.yml, for better rebase
2906c373c fixed appveyor badges
0d6dc079f update appveyor tokens
b44790cf9 Merge pull request #270 from inikep/dev
4235e5e25 updated visual\README.md
257b65a1d ToolsVersion="4.0"
200746427 removed _x64 and _x86
e583dd056 ImageHasSafeExceptionHandlers=false
eb160e1a2 appveyor.yml: added liblz4_x86.zip
45b8623af updated lib\dll\README.md
5625cc21e appveyor.yml: generate DLL package
c093e256d added files used by DLL package
be471c69d added fullbench-dll.sln that works with gcc/MinGW
f71340ce1 DLL dependencies moved to lib/dll/
f3f59bef4 updated appveyor.yml
b0e994cc5 upload executables to GitHub
d84a80e66 Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
923cf4447 bump version number
4c5c711b7 fixed minor conversion warning
f094f5314 silence a minor msan warning
0fe5c187a fixed minor msan warning
201149f8e fixed conversion warnings
fa158b411 fixed nameless union warnings
84af32d49 Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
5a5ebeec1 clang and g++ tests restricted to native only on Travis CI Ubuntu v12 environment
569897ddd Merge pull request #269 from inikep/dev
1b24cc115 fixed conversion warnings
9ad7508db Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
ecc55d19b fixed uintptr_t on pre-c99 compilers
3d3ccc154 Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
d5c9b39dd minor code refactoring
4bd9db1dd Merge pull request #268 from nemequ/dev
f42130ee4 cli : add --help command
84b97f90b fuzzer code refactoring fix g++ conversion warning
151021a3a cmake: add public domain dedication, and a note about the maintainer
b6bcb84ea cmake: make PIC optional when generating a static library
94542d845 made lz4hc explicitly dependent on lz4
bb97e8a6f fixed __GNUC__ macro
cb2230826 fixed fuzzer error
3580d9698 enabled deprecation warnings on remaining obsolete functions
874f3e095 update code comments
e528a82f0 Merge pull request #267 from terrelln/strict-aliasing
3d456bdf2 fixed conversion warnings
bf6dda69e updated frametest
85aeb0e4b Expose internal types to remove strict aliasing
dbfdd5131 Merge pull request #266 from terrelln/frame-ex
7aad356ff Add frame decompression example
ec6fb477b added visual\README.md
9d075580d Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
4f9db1383 updated xxhash lib
c72d2f5b1 minor typo
7d2e4b3cf Merge pull request #265 from terrelln/rm-zstd
5b37837e6 Fix license and remove references to zstd
1f246a989 Fixed#178 fullbench on small input
5e13a6ec4 improved man on compression level
f9c9bb364 Merge pull request #263 from terrelln/dict-ex
cd4462629 Merge pull request #264 from inikep/dev
52251d970 Using MinGW+MSYS to create DLL
72a4a84df udated README.md
2ecce6f15 lz4.h: added Introduction section
93e2477ca added lz4_manual.html
a340b734b added contrib/gen_manual
2278d1f02 custom block size
d57d3e4b4 better MSYS/MinGW support
74f127787 NB_LOOPS ?= -i1
cf8ca581d reordered Travis CI tests
767a23160 changed FUZZER_TIME
6a160b72e Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
924f00c86 fix empty body
94917c9a0 Add dictionary random access example
f34808e4a fixed some static analyzer warning
46f74d79b fixed sparse on stdout
fe48cef50 fixed multiples files on lz4cat (#184, by @libor-m)
1b1680594 fix unused variable warning
1b95d60c6 fixed synthetic benchmark mode slightly optimized test mode
acc48a3f8 fixed lz4cat changed lz4-test order
0c5e2e4d1 Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
fafba3973 updated NEWS
2987eb417 Merge pull request #262 from inikep/dev
063b4d92e fixed make install
773ae50b4 DLL exports only functions defined in liblz4.def
ca80a7c8f create DLL with MinGW/MSYS
67fd5dffd updated appveyor.yml
c794829a2 update .gitignore
9e7fe1062 removed <PlatformToolset>
dd14dca15 LZ4_DLL_IMPORT
325babddf added lz4.rc
ef2fdfe3d rename liblz4-dll to lz4-dll
7df658ec1 removed LZ4_DLL_EXPORT=1 (2)
4a6e7327c removed LZ4_DLL_EXPORT=1
bde0d340b added fullbench-dll.vcxproj
da48dbb57 Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
fe5148417 introduced LZ4_COMPILED_AS_DLL
d5903c8e5 Merge pull request #261 from terrelln/clean-fasttest
d594d9d87 Remove fasttest when cleaning
bd88e4007 Merge pull request #260 from nemequ/dev
661c48040 cmake: significant changes to accomodate a new "bundled" mode.
ce2202690 Merge pull request #259 from inikep/dev
96ea2e363 updated IntDir
39157c9f3 added lz4-dll project
ec63747de VS project: added IntDir
c6fd78f4f VS project: added setargv.obj
1d22fae1e added test-lz4-speed.py
85f861fc4 bench.c: support notification level
2c788b372 joined arm and aarch64 tests
0a22bf7d6 err0r(LZ4F_errorCodes code) uses ptrdiff_t
d06ab4f41 test powerpc64
230264913 tests\Makefile: added test-platform
c201327f1 added platformTest
0f73ed572 new Travis tests
90981a35c Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
c7ab95faa slightly improved HC compression ratio
e9089bcbe small compression ratio and speed improvement on small files
00c6370ad Fix#243 : detect and report fread() errors, by @iyokan
f6a7c6b57 Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
301c4e94e Merge pull request #258 from inikep/dev2
839085f05 Merge pull request #257 from inikep/dev
9b3dcb325 MOREFLAGS="-Werror"
7fbb163f3 added "make c_standards"
b6c407886 condensed travis.yml
2fbd7a512 updated travis.yml
ef54f9f9d added --rm option
d61bb74f2 updated declaration of LZ4F_freeDecompressionContext
7a98db266 added datagen.c to lz4.vcxproj
4b29af350 TreatWarningAsError=true only for debug
90d4401fe fixed appveyor.yml (3)
434d1230e fixed appveyor.yml (2)
58ead8dd2 fixed appveyor.yml
98be00847 Copy owner, file permissions and modification time
158156729 util.h requires CharacterSet=MultiByte
ef9d1b2d4 fixed --test
112499a73 Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
58d7a619a Merge pull request #254 from terrelln/dev
920bf2171 Fix LZ4_decompress_fast_continue() bug
207f478ba minor refactor
8195ba8f7 Updated man page
96565816b restricted deprecation warning to fairly recent gcc versions (>=6.0)
eaad740ac lz4frame obsolete enum should trigger deprecation warnings (on supported compilers)
079d5dd54 removed support for old error code enums from lz4frame_static.h, to free up namespace. note : lz4frame_static.h does not guaranteed API stability. note 2 : the macro to enable old error code enums is still present. Just needs to comment one line to re-enable them.
473acf5aa Merge pull request #252 from inikep/dev
10c5d5eee Merge pull request #253 from terrelln/exit-code
86a24c80f Fix typo in lz4 manpage
f30c56c08 Quiet gcc-4.6.3 narrowing warning
374090c7a Return error if input file does not exist.
fbede33fd fixed Travis tests
8ddaddc2d updated #include in util.h
f878c08b7 better correctness on big-endian 64-bits platforms
26dec498c small compression speed improvement on 64-bits systems
e8a6067e8 updated lz4cli.c
8b8c726a5 bench.c based on zstd
e06fcd903 redefined VOID for Windows/MSYS
e7648f4fc added -r option
6ebf8859e Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
4c496faba Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
6d6a3e0fb Merge pull request #250 from terrelln/test-mode
4c6610709 minor code refactoring
136caa552 Rename passThrough to testMode
eda8c4eb1 fix#198 : no longer requires to restart streaming decompression from where it stopped. It permits relocating remaining data into another memory space. Still, the same content must be presented.
04d728b87 Fix tests for test mode
4791cda04 fixed strict warnings
258a5e7fa updated comments
3dcafd35d Fix test mode and write to null
6f68a1692 fixed travis badges
99d11120b Merge pull request #249 from inikep/dev
46b49af37 fixed clang conversion warning
c66108ceb check fread() operation results (#243)
8847faffb updated programs/README.md
f38adde9e versionsTests moved to tests/
74ba377d7 updated .travis.yml
9f15a4c6e updated Visual Studio projects
84cedb463 updated links to LZ4 repository
e624978ea test programs moved to tests/
88cef262e documentation moved to doc/
b768da8ad Merge remote-tracking branch 'refs/remotes/lz4/dev' into dev
59adfd82d fix for ARM platform
67500142c Clarified license (#115, #244)
f17302769 moved cmake and debian directories to contrib (#245)
d05ca2ff6 fixed#247, reported by Felix Bolte
7e112cfa5 fixes static compilation under Windows (#246)
e64a92692 Merge pull request #242 from vapier/dev
92408d338 README: fix minor typo
52cdb4a5f Merge pull request #241 from vapier/dev
c5cb4aa52 Merge pull request #238 from kou/add-lz4lib-api
544abaced Merge pull request #240 from kou/add-missin-ext
43b631223 add a flag to disable static libs
829941e5a Add missing $(EXT) to lz4 program name
59771f41c Use different prefix for each file to avoid conflict
1ca408105 Merge pull request #239 from kou/use-lower-case-for-header-name
012c66536 Add LZ4LIB_API
46a7b94e8 Use lower case for header file name
3b9439ccd updated uninstall
8eae1320c Merge pull request #236 from inikep/dev
33993542a removed test artefacts
3eb143fff fix bug #232 at https://github.com/Cyan4973/lz4/issues/232
c8a78dba1 datagen depends on lz4.h for version string
53515b05e Merge pull request #235 from inikep/dev
92ee3e0ea fixed LZ4 repository links
39be6f16a versionTests require 32-bit gcc libraries
122fdd13a versionTests require python3
6adf7392a updated .travis.yml
8ac619afe Merge pull request #233 from inikep/dev
58a1616d7 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev
c77fdb44f disabled <RunCodeAnalysis> for Win32
28f98647e lz4 version source from lz4.h
dd51ca630 frametest depends on standard C time.h only
f327a8bdd fullbench depends on standard C time.h only
32a31663c fuzzer depends on standard C time.h only
64f556e61 lz4cli version number from lz4.h
5871585b6 coding style change
e5b0f1ea1 CFLAGS += $(MOREFLAGS)
829ae7587 remove EnablePREfast=true for Win32
547e1943c set OutDir
ab707e8b0 update IncludePath
3a7b4457a restored lz4.vcxproj
81d90609b added appveyor.yml
c200f7e09 renamed VS project
f4111cf04 Merge pull request #231 from inikep/dev
b3ae49c7b added <string.h> for strlen()
2cf32ef0f -b# and -e# options from zstd
c1864f69e fix mingw64 compilation warnings
7f08131f9 Merge pull request #226 from gsauthof/port-make
3fa60044e don't use variable eval syntax in if()
e1e3305db don't link the tools against the shared library
b0d42cb7e remove additional -fPIC switch because
681d5010d use gnu c99 to get both c99 and POSIX
584e1fe92 request POSIX.2001 definitions for fileno()
5bd3eaa48 minor comments
9578ff539 minor refactor
a7672dc4f minor decompression speed gains
182645ee0 lz4 as phony target
0214638eb Makefile : lz4 at root
63d22e7a1 minor comments clarifications
74e6afc5d minor comments clarification
2ac9ecec6 Fixed#157 : LZ4F_getFrameInfo() fails on valid null-content frame
45ea27949 Minor fuzzer code refactoring
de69eca47 Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
828332d23 fixed : fuzzer test, reported by @setharnold (#203)
b130fd592 minor refactoring changes
3c295ca4a Merge pull request #222 from kou/fix-build-error-with-mingw
0c63b7cbd Fix build error with MinGW on Linux
7316584ca Merge pull request #219 from BenWiederhake/dev
d3e2d8026 Use https wherever possible
cec38d1b7 Correctly reference self (github, not googlecode)
654003003 Use correct 'repository' hostname
91cce757f Updated xxhash library to v0.6.1
5540f4f93 stronger compilation flags
19e585cc4 modified : some more deprecated functions from lz4hc.h generate warnings
6b1600f41 changed : default benchmark LZ4_decompress_safe()
6923f3e23 Minor refactoring (code style)
410ec8154 Fixed : alignment warning
bc342ab94 minor refactoring
c8a573d32 very minor decompression speed gain
a5508b1ad fixed display result issue when benchmarking multiple files
e27e7316d minor code refactoring
2d7df8b30 Reduced bench dependency to standard C <time.h> Faster tests More accurate results on systems with high resolution clocks
8938e1074 minor compression speed improvement
c77c0c108 minor refactoring
3c0332600 minor refactoring (coding style)
1f47f3f74 Merge pull request #208 from Nimloth/dev
7bfbccb97 Add FreeBSD to install targets
5bb5d6ee1 Merge pull request #204 from irwand/dev
bc70a4b72 fixed non-C99 compilers
10e3eecc5 fixed non-C99 compiler
973baa2c1 Merge branch 'heapmode_alloc_failure_fix' into dev
3e5bb6753 alloc failure fix
8a42ee10a Merge pull request #201 from cernekee/cmake-pc
4bda09fc8 cmake: Install liblz4.pc
8b562a4f3 Merge pull request #197 from Cyan4973/fix-readme-travis
5cdf9a657 Fix wrong link for Travis
220e0b606 Merge pull request #194 from jzhuge/dev
cf6652f9b lz4cli: print library version
60ba8638c Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
219da951d Added zstd
2995a45e5 Merge pull request #185 from jwerner-chromium/dev
dc868cd5b Allow for safe in-place decoding
4fcb2e17f Remove whitespace from ends of lines
ad564c872 Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
db53bbb94 updated hyperlink
b84eb3349 Merge pull request #182 from kou/add-missing-ext
b8abffc39 Merge pull request #181 from kou/fix-build-failure-on-mingw
ab3e1d253 Add missing $(EXT)
14b40fb6d Fix build error on MinGW
62bbd102f Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
2785eeedf clarified man page as suggested in #170
631dc8f2b Merge pull request #173 from flandr/liblz4
d67ae8fe4 Merge pull request #172 from psteinb/add_static_cmake
4c9d851eb added static build target for OSX and Linux to cmake
495ea43d9 Merge pull request #162 from KyleJHarper/20151021_examples
7aa004377 Forgot to make clean before commiting. Removing binary.
7f96babc4 Several changes to address a few concerns from Yann. See Google Group LZ4c topic 'Reusing compression/decompression resources' for details.
d1be164e4 Added a note about why char* is used. Also removed a binary that wasn't supposed to be in there.
0d2518567 Finished the two example files. Decided to avoid adding anything to lz4.c/h to expose LZ4_compress_generic().
02be6631b Took out the basics and placed them into basics.c. Added decompression and a wrapper for the generic call. I will likely break this file up into 2 examples before submission.
67b3a2470 Final tests and reporting are done. As expected there isn't much to be gained by jumping the chain. In most of my tests I did see a moderate performance gain when invoking LZ4_compress_generic() directly with normal text. This could very easily be an edge case. Either way it's interesting and worth sharing.
bdd9143e0 Didn't add my file for some reason. Odd.
73dd936b9 Wrote the skeleton of the performance test. Need to finish tests for the rest of the call stack and then my own copy of generic().
2c8013812 heapmode macro option for lz4hc
ff3324b8d Fix : compiler-independent macro to remove deprecation warnings
64f8b037e clarified lz4 license
2df602568 Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
7d4575e54 fixed link typo
012ab2f52 Merge pull request #152 from dcolascione/dev
94a0954b1 Remove .dir-locals.el
4f1ff58f8 Tweak error message
975c06b1a Improve error messages for attempted console IO
6f1fe1b70 Add .dir-locals.el for setting tab-width
0011033e3 fixed#150 : blank lines when converting lz4.1 to .ps or .pdf
248553c5e minor : show in table that frame checksum is optional
cd18ed308 create link to dynamic library without using `cp -a` (#147)
fb6fece77 Updated Makefile and .travis
8a750aa35 Add static library target for msbuild
13f12aa02 Wrapped possibly empty string (Emscripten) with quotes (@Fordi)
4a5a003c4 updated cli help
9aa5504f0 New unalign access method
e5aee601e updated xxhash
e64345506 Merge pull request #142 from bioothod/master
d15ac1f1a Slightly improved compression speed on ARMv6
0f2bf0c54 Improved performance on ARMv6
fb4d3ef2c Added .gitignore
4b4384772 Updated xxhash to r41
2a6ad2a7f debian: changed name lz4->liblz4
2f3e4bf32 debian: changed name lz4->liblz4
31fed3167 debian: changed name lz4->liblz4
309d1e087 debian: remove sbin dir
2131eedf7 debian: renamed lz4->liblz4, updated installed library files
b47786b9e debian: updated debianization to cdbs
a8f33076a cmake: when building library make it shared
33fa0bfb7 Added debianization
b4b739c4b cmake: bump version to the latest release
5dd12b4a7 faster sanitize tests
60119e258 Makefile : generates *.o for faster processing
9517bd923 minor simplification lz4_readarch()
a60d11e46 update on decompression speed
00eedb37d minor refactor
3e579afa6 Slightly improved decompression speed Slightly decreased decompression stack usage in 64-bits mode
795a2cdb1 sligthly improved decompression speed
a3c6daba6 liblz4 : Moved XXH_NAMESPACE to CPPFLAGS
d86dc9167 Merge pull request #131 from Cyan4973/dev
dfabec2d1 Fixed a few minor visual analyzer warnings
4abc86e0b Increased stack memory considered normal by Visual static analyzer
db59c7d33 Merge pull request #130 from Cyan4973/dev
8901dcdc9 removed one malloc
1098d897a lz4io : removed one malloc
81f19264a lz4cli : add one malloc test
aa1350e13 lz4hc : added one malloc test
6fe48b918 changed datagen to remove one malloc
15603d956 Added static analyzer test in Visual
48feb909d Merge pull request #129 from Cyan4973/dev
57e14d4c1 renamed directory to versionsTest
14064beaf Updated NEWS
7273da38b minor example clarification
0cc38f9eb Updated lib readme
454d109c5 Fixed : fuzzer display messages
caba2cc0f staticAnalyze use same compilation flags as normal build
8e8dc17ca Ensured make test is not confused with directory test
192ee724a Added namespace ability to xxhash
c04df7e1b Changed : static library is no longer compiled with -fPIC by default (this option can still be added on the command line) See #53
cdef03389 Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
992faef5a Merge pull request #124 from toddlipcon/master
3d9adf7af Merge pull request #122 from keszybz/master
02ff90ac9 Merge pull request #121 from keszybz/add-gitignore
91138150c Fix cmake build to use gnu flags on clang
2ce2e6008 Minor doc clarification on sparse mode support
b17d3220d Minor lib readme update
3bcf68ff3 Use lz4 binary to check output
386d216cd Add example which uses the frame api through the library
0e0c32dfb Make git ignore generated files
7a532eedd Minor update on source ports
8a7bcead9 Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
1cec6cb03 Updated badges
7f3d82def Merge pull request #114 from lpsantil/djgpp
f0a0e4430 READM fixup
a3a479fc2 djgpp port now using the proposed contrib structure
00b6535b4 Reverted all `Makefile` changes
936f75b4d Renamed the djgpp README so that it gets converted to HTML by github. Fixed breakage of `make install`.
81d6b5634 Makefile cleanups to make building with Andrew Wu's build-djgpp cross compilers (and everyone else) simpler.
faa0bce4e Fixup lib/Makefile so that is builds a library for djgpp using Andrew Wu's build-djgpp cross compilers
b2bbb2790 Fixup programs/{Makefile,lz4cli.c,lz4io.c} to build with Andrew Wu's build-djgpp cross compilers. Add some documentation in README.DJ
git-subtree-dir: src/lz4
git-subtree-split: c10863b98e1503af90616ae99725ecd120265dfb
b02bfa754 Tag open source release 1.1.7.
824e6718b Add a loop alignment directive to work around a performance regression.
55924d110 Add GNUInstallDirs to CMake configuration.
632cd0f12 Use 64-bit optimized code path for ARM64.
77c12adc1 Add unistd.h checks back to the CMake build.
c8049c582 Replace getpagesize() with sysconf(_SC_PAGESIZE).
18e2f220d Add guidelines for opensource contributions.
f0d3237c3 Use _BitScanForward and _BitScanReverse on MSVC.
71b8f8688 Add SNAPPY_ prefix to PREDICT_{TRUE,FALSE} macros.
be6dc3db8 Redo CMake configuration.
e4de6ce08 Small improvements to open source CI configuration.
c756f7f5d Support both static and shared library CMake builds.
038a3329b Inline DISALLOW_COPY_AND_ASSIGN.
a8b239c3d snappy: Remove autoconf build configuration.
27671c6ae Clean up CMake header and type checks.
548501c98 zippy: Re-release snappy 1.1.5 as 1.1.6.
513df5fb5 Tag open source release 1.1.5.
5bc9c82ae Set minimum CMake version to 3.1.
e9720a001 Update Travis CI config, add AppVeyor for Windows CI coverage.
f24f9d2d9 Explicitly copy internal::wordmask to the stack array to work around a compiler optimization with LLVM that converts const stack arrays to global arrays. This is a temporary change and should be reverted when https://reviews.llvm.org/D30759 is fixed.
82deffcde Remove benchmarking support for fastlz.
18488d621 Use 64 bit little endian on ppc64le.
7b9532b87 Improve the SSE2 macro check on Windows.
7dadceea5 Check for the existence of sys/uio.h in autoconf build.
83179dd8b Remove quicklz and lzf support in benchmarks.
c8131680d Provide a CMakeLists.txt.
ed3b7b242 Clean up unused function warnings in snappy.
8b60aac4f Remove "using namespace std;" from zippy-stubs-internal.h.
7d7a8ec80 Add Travis CI configuration to snappy and fix the make build.
1cd3ab02e Rename README to README.md. It already in markdown, we might as well let github know so that it renders nicely.
597fa795d Delete UnalignedCopy64 from snappy-stubs since the version in snappy.cc is more robust and possibly faster (assuming the compiler knows how to best copy 8 bytes between locations in memory the fastest way possible - a rather safe bet).
039b3a7ac Add std:: prefix to STL non-type names.
3c706d223 Make UnalignedCopy64 not exhibit undefined behavior when src and dst overlap.
d3c6d20d0 Add compression size reporting hooks.
626e1b9fa Use #ifdef __SSE2__ for the emmintrin.h include, otherwise snappy.cc does not compile with -march=prescott.
2d99bd14d 1.1.4 release.
8bfb028b6 Improve zippy decompression speed.
818b58338 adds std:: to stl types (#061)
27c5d8652 Re-work fast path for handling copies in zippy decompression.
4a7409408 Speed up Zippy decompression in PIE mode by removing the penalty for global array access.
38a5ec5fc Re-work fast path that emits copies in zippy compression.
094c67de8 Speed up the EmitLiteral fast path, +1.62% for ZFlat benchmarks.
fce661fa8 Speed up zippy decompression by removing some zero-extensions.
e788e527d Avoid calling memset when resizing the buffer.
32d6d7d8a Merge pull request #6 from deviance/provide-pkg-config-data
971613510 Add #ifdef to guard against macro redefinition if this is included in another Google project that also defines this.
0000f997d Merge pull request #13 from huachaohuang/patch-1
d53de1879 Make heuristic match skipping more aggressive.
2b9152d9c Default to glibtoolize instead of libtoolize if it exists, and also make it customizable through the environment variable $LIBTOOLIZE.
0800b1e4c Work around an issue where some compilers interpret <:: as a trigraph. Also correct the namespace name.
e7d2818d1 Unbreak the open-source build for ARM due to missing ATTRIBUTE_PACKED declaration.
7525a1600 Fix an issue where the ByteSource path (used for parsing std::string) would incorrectly accept some invalid varints that the other path would not, causing potential CHECK-failures if the unit test were run with --write_uncompressed and a corrupted input file.
ef5598aa0 Make UNALIGNED_LOAD16/32 on ARMv7 go through an explicitly unaligned struct, to avoid the compiler coalescing multiple loads into a single load instruction (which only work for aligned accesses).
b8cd908a8 Allow to compile in nested packages.
96a2e340f Update URLs in the Snappy README to reflect the move to GitHub.
0852af760 Move the logic from ComputeTable into the unit test, which means it's run automatically together with the other tests, and also removes the stray function ComputeTable() (which was never referenced by anything else in the open-source version, causing compiler warnings for some) out of the core library.
d80342922 Fix signed-vs.-unsigned comparison warnings.
d2cb73b6a Provide pkg-config data
efb39e81b Release Snappy 1.1.3; getting the new Uncompress variant in a release is nice, and it's also good to finally get an official release out after the migration to GitHub.
eb66d8176 Initialized members of SnappyArrayWriter and SnappyDecompressionValidator. These members were almost surely initialized before use by other member functions, but Coverity was warning about this. Eliminating these warnings minimizes clutter in that report and the likelihood of overlooking a real bug.
b2312c4c2 Add support for Uncompress(source, sink). Various changes to allow Uncompress(source, sink) to get the same performance as the different variants of Uncompress to Cord/DataBuffer/String/FlatBuffer.
b2ad96006 Changes to eliminate compiler warnings on MSVC
e7a897e18 Fixed unit tests to compile under MSVC.
86eb8b152 Change a few branch annotations that profiling found to be wrong. Overall performance is neutral or slightly positive.
11ccdfb86 Sync with various Google-internal changes.
22acaf438 Change some internal path names.
git-subtree-dir: src/snappy/snappy
git-subtree-split: b02bfa754ebf27921d8da3bd2517eab445b84ff9
* Update unity build for RocksDB changes
* Log RocksDB options on startup
* Support RocksDB option strings
* Support full file bloom filters
You can now configure most RocksDB options with RocksDB's option
string scheme.
Set "filter_full" to 1 to make bloom filters for an
entire file rather than each block. More memory will be
needed during compaction but less memory will be needed
during fetching for large databases. Does nothing unless
bloom filters are enabled with "filter_bits".
Example:
options = max_compaction_bytes=64;max_bytes_for_level_multiplier=64
clock_cache_mb = 96
filter_bits = 10
filter_full = 1
Do not dispatch a transaction received from a peer for
processing, if it has already been dispatched within the
past ten seconds.
Increase the number of transaction handlers that can be in
flight in the job queue and decrease the relative cost for
peers to share transaction and ledger data.
Additionally, make better use of resources by adjusting the
number of threads we initialize, by reverting commit
68b8ffdb63.
aead40417 fix HISTORY.md typo
a0cdc3cec Bump version to 5.8.7
7513f6350 Fix IOError on WAL write doesn't propagate to write group follower
9e47084ce Bump version to 5.8.6
36074ba5d Enable cacheline_aligned_alloc() to allocate from jemalloc if enabled.
aa00523e0 Add -DPORTABLE=1 to MSVC CI build
cf2b98237 Bump version to 5.8.5
e8c9350f2 Blob DB: not using PinnableSlice move assignment
4907d2463 Bump version to 5.8.4
5d928c795 Blob DB: Fix race condition between flush and write
725bb9d66 Blob DB: Fix release build
b7367fe84 Bump version to 5.8.3
13b2a9b6f Blob DB: use compression in file header instead of global options
5dc70a15c Fix PinnableSlice move assignment
9019e9125 dynamically change current memtable size
7f1815c37 Bump version to 5.8.2
2584a18ef Blob DB: Fix BlobDBTest::SnapshotAndGarbageCollection asan failure
17f67b546 PinnableSlice move assignment
6fb56c582 Blob DB: Add compaction filter to remove expired blob index entries
f90ced92f Blob DB: fix snapshot handling
632f36dcd Blob DB: option to enable garbage collection
11bacd578 Blob DB: Fix flaky BlobDBTest::GCExpiredKeyWhileOverwriting test
f98efcb1e Blob DB: Evict oldest blob file when close to blob db size limit
c1e99eddc Blob DB: cleanup unused options
ffc3c62ca Blob DB: Initialize all fields in Blob Header, Footer and Record structs
9e8254090 Blob DB: update blob file format
d66bb21e1 Blob DB: Inline small values in base DB
05d5c575a Return write error on reaching blob dir size limit
2b8893b9e Blob DB: Store blob index as kTypeBlobIndex in base db
419b93c56 Blob DB: not writing sequence number as blob record footer
8afb0036c fix lite build
dded348dd Blob DB: Move BlobFile definition to a separate file
374736123 add GetLiveFiles and GetLiveFilesMetaData for BlobDB
8cff6e945 Enable WAL for blob index
c29347290 Add ValueType::kTypeBlobIndex
eae53de3b Make it explicit blob db doesn't support CF
65aec19df Fix memory leak on blob db open
30b38c98c TableProperty::oldest_key_time defaults to 0
2879f4beb Bump version to 5.8.1
88595c882 Add DB::Properties::kEstimateOldestKeyTime
266ac245a Bumping version to 5.8
64185c23a update HISTORY.md for DeleteRange bug fix
e83d6a02e Not using aligned_alloc with gcc4 + asan
0980dc6c9 Fix wrong smallest key of delete range tombstones
b76797231 avoid use-after-move error
c41744270 CMake: Fix formatting
c21ea8f7a CMake: Add support for CMake packages
544434558 add Erlang to the list of language bindings
2972a702a Minor updates to FlushWAL blog
fbfa3e7a4 WriteAtPrepare: Efficient read from snapshot list
b01f426f5 Blog post for FlushWAL
503db684f make blob file close synchronous
3c840d1a6 Allow DB reopen with reduced options.num_levels
92bfd6c50 Fix DropColumnFamily data race
7fdf735d5 Pinnableslice examples and blog post
7fbb9ecca support disabling checksum in block-based table
19cc66dc4 fix clang bug in block-based table reader
7eba54eb9 test compaction input-level split range tombstone assumption
cd26af347 Add unit test for WritePrepared skeleton
a12479819 Improved transactions support in C API
c10b39131 LANGUAGE-BINDINGS.md: add another rust binding
90177432e Remove leftover references to phutil_module_cache
234f33a3f allow nullptr Slice only as sentinel
ccf7f833e Use PinnableSlice in Transactions
1dfcdb15f Extend pin_l0 to filter partitions
39ef90055 stop calling memcmp with nullptrs
78cb6b611 Provide byte[] version of SstFileWriter.merge to reduce GC Stall
867fe92e5 Scale histogram bucket size by constant factor
f004307e9 CMake improvements
09ac6206a Circumvent ASAN false positive
5b68b114f Blob db create a snapshot before every read
4624ae52c GC the oldest file when out of space
8ace1f79b add counter for deletion dropping optimization
0d8e992b4 Revert the mistake in version update
5358a8056 add VerifyChecksum to HISTORY.md
ed0a4c93e perf_context measure user bytes read
1efc600dd Preload l0 index partitions
bddd5d363 Added mechanism to track deadlock chain
c1384a707 fix db_stress uint64_t to int32 cast
29877ec7b Fix blob db crash during calculating write amp
8f2598ac9 Enable Cassandra merge operator to be called with a single merge operand
9a44b4c32 Allow merge operator to be called even with a single operand
ac8fb77af fix some misspellings
23593171c minor improvements to db_stress
af012c0f8 fix deleterange with memtable prefix bloom
1c8dbe2aa update scores after picking universal compaction
eb6425303 Update WritePrepared with the pseudo code
132306fbf Remove PartialMerge implementation from Cassandra merge operator
71598cdc7 Fix false removal of tombstone issue in FIFO and kCompactionStyleNone
3204a4f64 Fix missing stdlib include required for abort()
7aa96db7a db_stress rolling active window
dfa6c23c4 Update RocksDBCommonHelper to use escapeshellarg
e367774d1 Overload new[] to properly align LRUCacheShard
ad42d2fcb Remove residual arcanist_util directory
279296f4d properly set C[XX]FLAGS during CMake configure-time checks
c5f0c6cc6 compile with correct flags to determine SSE4.2 support
185ade4c0 cmake: support more compression type
5449c0990 rocksdb: make buildable on aarch64
a144a9782 Fix for CMakeLists.txt on Windows for RocksJava
acf935e40 fix deletion dropping in intra-L0
8254e9b57 make sst_dump compression size command consistent
74f18c130 db_bench support for non-uniform column family ops
5de98f2d5 approximate histogram stats to save cpu
3f5888430 Fix c_test ASAN failure
e5a1b727c Fix blob DB transaction usage while GC
6f051e0c7 fix corruption_test valgrind
ac098a462 expose set_skip_stats_update_on_db_open to C bindings
666a005f9 Support prefetch last 512KB with direct I/O in block based file reader
ad77ee0ea Revert "Makefile: correct faligned-new test"
b87ee6f77 Use more keys per lock in daily TSAN crash test
25df24254 Add column families related functions (C API)
64f848435 block_cache_tier: fix gcc-7 warnings
0cecf8155 Write batch for `TransactionDB` in C API
6a9de4347 Windows.h macro call fix
23c7d1354 fix comment
1fbad84b6 Makefile: correct faligned-new test
7848f0b24 add VerifyChecksum() to db.h
47ed3bfc3 fix WinEnv assertions
d97a72d63 Try to repair db with wal_dir option, avoid leak some WAL files
36375de76 gcc-7/i386: markup intentional fallthroughs
bdc056f8a Refactor PessimisticTransaction
a9a4e89c3 Fix valgrind complaint about initialization
4ca11b4b0 Update USERS.md
c9804e007 Refactor TransactionDBImpl
20dc5e74f Optimize range-delete aggregator call in merge helper.
0d4a2b733 Avoid blob db call Sync() while writing
627c9f1ab Don't add -ljemalloc when DISABLE_JEMALLOC is set
dce6d5a83 db_bench background work thread pool size arguments
4f81ab38b Makefile: fix for GCC 7+ and clang 4+
92afe830f Update all blob db TTL and timestamps to uint64_t
5883a1ae2 Fix /bin/bash shebangs
cc01985db Introduce bottom-pri thread pool for large universal compactions
0b814ba92 Allow concurrent writes to blob db
2c45ada4c Blob DB garbage collection should keep keys with newer version
58410aee4 Fix the overflow bug in AwaitState
c3d5c4d38 Refactor TransactionImpl
060ccd4f8 support multiple CFs with OPTIONS file
345387067 Fix statistics in RocksJava sample
1900771bd Dump Blob DB options to info log
3218edc57 Fix universal compaction bug
6a36b3a7b fix db get/write stats
a84cee812 Add a missing "once" in .h
21696ba50 Replace dynamic_cast<>
e85f2c64c Prevent empty memtables from using a lot of memory
ac748c57e Fix FIFO Compaction with TTL tests
aaf42fe77 Move blob_db/ttl_extractor.h into blob_db/blob_db.h
aace46516 Fix license headers in Cassandra related files
50a969131 CacheActivityLogger, component to log cache activity into a file
6083bc79f Blob DB TTL extractor
710411aea fix asan/valgrind for TableCache cleanup
3a3fb00b7 TARGETS file not setting sse explicitly
fca4d6da1 Build fewer tests in Travis platform_dependent tests
8f553d3c5 remove unnecessary internal_comparator param in newIterator
7f6d012d7 "ccache -C" in Travis
d12691b86 move TableCache::EraseHandle outside of db mutex
f33f11368 fix db_bench argument type
e7697b8ce Fix LITE unit tests
3ce20e985 Fix use of RocksDBCommonHelper in cont_integration.sh
c281b4482 Revert "CRC32 Power Optimization Changes"
9980de262 Fix FIFO compaction picker test
2289d3811 CRC32 Power Optimization Changes
30b58cf71 Remove the orphan assert on !need_log_sync
fe1a5559f Fix flaky write_callback_test
addbd279c 5.6.1 release blog post
30edff308 buckification: remove explicit `-msse*` compiler flags
2b259c9d4 Lower num of iterations in DeadlockCycle test
277f6f23d Release note for partitioned index/filters
5e731a138 Remove unused rocksdb arcanist lib
9b11d4345 Fix broken links
06f191744 add vcpkg as an windows option
ea8ad4f67 Fix compaction div by zero logging
34112aeff Added db paths to c
1d8aa2961 Gcc 7 ParsedInternalKey replace memset with clear function.
a4c42e800 Fix UBSAN issue of passing nullptr to memcmp
16e038820 LRUCacheShard cache line size alignment
216644c61 enable UBSAN macro in TARGETS
e67b35c07 Add Iterator::Refresh()
a34b2e388 Fix caching of compaction picker's next index
72502cf22 Revert "comment out unused parameters"
1d7048c59 comment out unused parameters
534c255c7 Cassandra compaction filter for purge expired columns and rows
63163a8c6 Remove make_new_version.sh
0302da47a Reduce blob db noisy logging
3e5ea29a8 Fix Flaky DeleteSchedulerTest::ImmediateDeleteOn25PercDBSize
a22b9cc6f overlapping endpoint fixes in level compaction picker
ffd2a2eef delete ExpandInputsToCleanCut failure log
3e6e863b1 Remove arcanist_util directory
36651d14e Moving static AdaptationContext to outside function
6e3ee015f Update java/rocksjni.pom
ecff9d5e3 Include write_buffer_manager in ImmutableDBOptions::Dump
ae28634e9 Remove some left-over BSD headers
33b1de82a Remove format compatibility hack
2f375154e checkout local branch in check_format_compatible.sh
ddb22ac59 avoid collision with master branch in check format
0c03a7f17 set the remote for git checkout
7ac184c6d Revert cmake -DNDEBUG for non-MSVC
0655b5858 enable PinnableSlice for RowCache
00464a314 Fix column_family_test with LITE build
b2dd192fe tools/write_stress.cc: Correct "1204" typos.
cbaab3044 table/block.h: change memset
f1a056e00 CodeMod: Prefer ADD_FAILURE() over EXPECT_TRUE(false), et cetera
4a2e4891f Add back the LevelDB license file
a7321fc97 Remove the licensing description in CONTRIBUTING.md
3c327ac2d Change RocksDB License
132013366 Make TARGETS file portable
ccf5f08f8 Set CACHE_LINE_SIZE for s390, PPC, ARM64
67510eeff db_crashtest.py: remove need for shell
4267eb00d Remove punit tests
5bfb67d90 Enable write rate limit for updaterandom benchmark
20a691d98 Update HISTORY to release 5.7
98d1a5510 db_bench to by default verify checksum
26ce69b19 Update blob db to use ROCKS_LOG_* macro
43e4eef77 remove unnecessary fadvise
21b17d768 Fix BlobDB::Get which only get out the value offset
70440f7a6 Add virtual func IsDeleteRangeSupported
7550255c5 Add JAVAC_ARGS as Makefile variable
7a0b5de77 Gcc 7 ignored quantifiers
000bf0af3 Improve the design and native object management of Stats in RocksJava
269d383d5 Bump version to 5.7
c32f27223 Fixes db_bench with blob db
fcd99d27c db_bench_tool: fix buffer size
87128bd5c fix regression test
8f927e5f7 Fix undefined behavior in Hash
643b787c7 Added a note about LZ4 compression dependency
56656e12d Temporarily disable FIFOCompactionWithTTLTest
b5fb85ec5 fix valgrind init complaint
657df29ea Add max_background_jobs to db_bench
a43c053ad remove duplicated utilities/merge_operators/cassandra/test_utils.cc in src.mk
7c4a9e6c9 Initialize a variable in ldb to make code analysis tool happy
98669b535 init filters_in_partition_
0013bf14e fix asan and valgrind leak report in test
521b4c28b rocksdb 5.5.1 release post
33042573d Fix GetCurrentTime() initialization for valgrind
f6b9d9355 Fix clang error in PartitionedFilterBlockBuilder
45b9bb033 Cut filter partition based on metadata_block_size
f4ae1bab0 update history for OnBackgroundError and DeleteRange fix
1cb8c6de6 Add -enable_pipelined_write to db_bench and add two defaults
7604b463b Update the AddDBStats in LITE
1e34d07e1 Simplify and document sync rules for logs_ etc
d310e0f33 Regression test for empty dedicated range deletion file
e9f91a517 Add a fetch_add variation to AddDBStats
c1b375e96 skip generating empty sst
67b417d62 fix format compatible test
afbef6518 Bug fix: Fast CRC Support printing is not honest
397ab1115 Improve Status message for block checksum mismatches
18c63af6e Make "make analyze" happy
01534db24 Fix the reported asan issues
1cd45cd1b FIFO Compaction with TTL
982cec22a Fix TARGETS file tests list
b49b37109 allow numa >= 2.0.8
e517bfa2c CLANG Tidy
dc3d2e4d2 update compatible test
89468c01d Fix Windows build broken by 5c97a7c0664d4071768113814e9ba71fe87e18cf
51778612c Encryption at rest support
7061912c2 Trivial typo in HISTORY.md
2a9cd8799 Fix jni WriteBatchThreadedTest
0025a3640 revert perf_context and io_stats to __thread
5c97a7c06 Unit Tests for sync, range sync and file close failures
4cee11f4e Intra-L0 blog post
857e9960b Improve the error message for I/O related errors.
d757355cb Fix bug that flush doesn't respond to fsync result
8e6345d2d Update rename of ParanoidCheck
499ebb3ab Optimize for serial commits in 2PC
0ac4afb97 Sanitize partitioning options
521724ba8 fixed wrong type for "allow_compaction" parameter
71f5bcb73 Introduce OnBackgroundError callback
88cd2d96e Downgrade option sanitiy check level for prefix_extractor
6837a1762 Fix Data Race Between CreateColumnFamily() and GetAggregatedIntProperty()
af1746751 WriteBufferManager will not trigger flush if much data is already being flushed
9467eb614 Fix flush assertion with tsan
048446fc7 Fix cassandra ASAN use-after-free
a21db161c Implement ReopenWritibaleFile on Windows and other fixes
c430d69ee fix coredump for release nullptr
0d278456c default implementation for InRange
cbd825dee Create a MergeOperator for Cassandra Row Value
2c98b06bf Remove pin_slice option by making it the default
c80c6115d add db_bench options for partitioning
6a3377f45 Synchronize statistic enumeration values between statistics.h and java API
53dda8797 Do not run RateLimiterTest.Rate test on Travis+Mac OSX.
ae8571f5c Fix blob db compression bug
7a380deff Update blob_db_test
89ad9f3ad Allow ignoring unknown options when loading options from a file
6b5a5dc5d fixed typo
0f228be3b fixed typo in util/dynamic_bloom.h
c217e0b9c Call RateLimiter for compaction reads
91e2aa3ce write exact sequence number for each put in write batch
6f4154d69 record index partition properties
5d5a28a98 Fix Clang release build broken by 5582123dee8426a5191dfd5e846cea8c676c793c
0175d58c3 Make direct I/O write use incremental buffer
7a270069b GNU C library for struct tm has 2 additional fields.
d713471da Limit trash directory to be 25% of total DB
9bb91e932 Dedup release
27b450165 Update HistogramTypes in the Java API
e97304c68 update history for 5.6
5582123de Sample number of reads per SST file
db818d2d1 Fix RocksDB Lite build with CLANG
a472c4ae4 update 5.5 change log
bc09c8a0d Fix crash in PosixWritableFile::Close() when fstat() fails
6d0f22e42 Fix mock_env.cc uninitialized variable
c2012d488 Java APIs for put, merge and delete in file ingestion
85dace2af Disable DBRangeDelTest::TailingIteratorRangeTombstoneUnsupported for ubsan
d4f7731b6 fix travis error with init time in mockenv
550a1df72 Fix clang errors by asserting the precondition
cc5f9339e Fix concurrency issue with filter_block_set_
2e64f450d bump version to 5.6
afbc2d0d2 Force travis to build with clang on MacOS
b172a3f1a Fix warnings while generating RocksJava documentation
52a7f38b1 WriteOptions.low_pri which can throttle low pri writes if needed
26a8a8071 Switch from CentOS 5 to CentOS 6 for crossbuilding RocksJava
dba9f3722 Fix db_write_test clang/windows build failure
c7662a44a fixed typo
7e8d95cc0 Fix the Java build which was broken by a4d9c02
7e5fac2c3 remove test dir before exit when current regression is running
7f6c02dda using ThreadLocalPtr to hide ROCKSDB_SUPPORT_THREAD_LOCAL from public…
138b87eae Fix interaction between CompactionFilter::Decision::kRemoveAndSkipUnt…
95b0e89b5 Improve write buffer manager (and allow the size to be tracked in block cache)
a4d9c0251 Pass CF ID to MemTableRepFactory
f68d88be5 Fix DBWriteTest::ReturnSequenceNumberMultiThreaded data race
215076ef0 Fix TSAN: avoid arena mode with range deletions
3a8a848a5 account for L0 size in estimated compaction bytes
0fae3f5dd codemod: format TARGETS with buildifier [5/5] (D5092623)
872199606 add checkpoint support for single db in regression test
5a9b4d743 Retire memenv https://github.com/facebook/rocksdb/pull/2082
d6019651b sync internal/external TARGETS
bbaba51bb Add missing index type to C-API
292edfd51 travis: test with xcode8.3 (OS X 10.12)
0dc3040d5 db: avoid `#include`ing malloc and jemalloc simultaneously
9b3ed8350 fix regression test
9c9909bf7 Support ingest file when range deletions exist
ad19eb868 Fixing blob db sequence number handling
51ac91f58 Histogram of number of merge operands
345878a7f update blob_db_test
cbc821c25 change regression rebuild to one level
103d0692e Avoid unsupported attributes when not building with UBSAN
5fd04566c travis: reduce the number of travis builders
2d05002b9 RocksDB 5.4.5 release blog post
7eca90f18 Update RocksDB blog authors
d03c34497 update comment of GetNextFile
f7bb1a006 support merge and delete in file ingestion
c2c62ad4e Reorder variables of ReadOptions
7bb1f5d48 Increase of compaction threads should be logged at info level instead of a warning
6c456ecae Clean zstd files
347e16f83 codemod: replace `headers = AutoHeaders.*` with `auto_headers`
0be636bf7 Fix db_bench build break with blob db
135ee6a3f fix tsan crash data race
a99fb9928 fix column_family_test asan
f41bffb3d travis: clang-3.6 -> clang-4.0
e7612798b update buckifer/TARGETS
bb01c1880 Introduce max_background_jobs mutable option
5a2530462 Fix the CMakeLists for RocksJava
41cbb7274 options.delayed_write_rate use the rate of rate_limiter by default.
506803466 range sync should be enabled
02594b5f1 Fix build errors in blob_dump_tool with GCC 4.8
52d9e5f7b Fix column family seconds_up accounting
7d8207f1f Fix errors in clang-analyzer builds
85b8569ae Fix release build on Linux
69ec8356b Allow SstFileWriter to use the rate limiter
6cc9aef16 New API for background work in single thread pool
9d0a07ed5 Fix rocksdb.estimate-num-keys DB property underflow
578fb0b1d Simple blob file dumper
ac39d6bec Core-local statistics
3e86c0f07 disable direct reads for log and manifest and add direct io to tests
15ba4d6c4 Address MS Visual Studio 2017 issue with autovector
88c818e43 Replace deprecated RocksDB#addFile with RocksDB#ingestExternalFile
228f49d20 Fix data races caught by tsan
4c9d2b104 remove #include port/port.h in public header file
07bdcb91f New WriteImpl to pipeline WAL/memtable write
d746aead1 Suppress clang-analyzer false positive
217b866f4 column_family_test: EnvCounter::num_new_writable_file_ to be atomic
9f839a7f6 keep util/build_version.cc when make clean
7eecd40a4 add emacs tags file - etags
9bbba4fec Remoe unused BlockBasedTable::compaction_optimized_
f5ba131bf Fixed some spelling mistakes
146b7718f Fix mingw compilation with -DNDEBUG
a36220ccf fix unity test
0ebdd7057 fixed typo
8032f4cb3 Remove -pie in TSAN
362ba9b02 Release RocksDB 5.5.0
ba685a472 Support ingest_behind for IngestExternalFile
01ab7b528 Add ROCKSDB_LIBRARY_API macro to a few C APIs, to fix windows build
cb9392a09 add Transactions and Checkpoint to C API
445f1235b s/std::snprintf/snprintf
cd593c283 Fix travis java_test
f720796e2 fixed typo
a48a62d5b define NDEBUG in CMake non-debug builds
1ca723dbd C API: support pinnable get
2ef15b85d Core-local stats blog post
4f9e69ccf fix log err
11c5d4741 cross-platform compatibility improvements
d00433302 Put lib files into suitable path in RPM package
86d549253 Fix build error with blob DB.
254c46800 Fix the RocksJava Release on Windows
7a47b431f Fix .gitignore pattern
fa5a15ceb Make sure that zstd is statically linked correctly in the Java static build
3fa9a39c6 Add GetAllKeyVersions API
1a60982a5 Simplified instructions for CentOS
a5cc7ecec Facility for cross-building RocksJava using Docker
ccd3dddf5 Blog post for partitioned index/filters
b145c34d7 Update blog authors
bbe9ee7dd core-local array type conversions
c2be43430 Build and link with ZStd when creating the static RocksJava build
c61e72c42 Add missing files of blob_db to CMake file
3907c94ff Fix ColumnFamilyTest:BulkAddDrop
cda5fde2d CoreLocalArray class
93949667c update TARGETS
4e83b8001 title: Bulkoading -> title: Bulkloading
d85ff4953 Blob storage pr
492fc49a8 fix readampbitmap tests
be421b0b1 portable sched_getcpu calls
0f559abdb Add NO_UPDATE_BUILD_VERSION option to makefile
3a04a254a Flink state
35df23fe8 Fix suite exclisions
e7cea86f7 Fixes the CentOS 5 cross-building of RocksJava
259a00eac unbiase readamp bitmap
a62096696 port: updated PhysicalCoreID()
df035b682 Print compaction_options_universal.stop_style in LOG file
4897eb250 dont skip IO for filter blocks
3f73d54bb Add C API to set max_file_opening_threads option
0b69e5079 Define CACHE_LINE_SIZE only when it's not defined
2cd00773c Add bulk create/drop column family API
40af2381e Object lifetime in cache
fdaefa030 travis: add Windows cross-compilation
a30a69603 do not read next datablock if upperbound is reached
2d42cf5ea Roundup read bytes in ReadaheadRandomAccessFile
264d3f540 Allow IntraL0 compaction in FIFO Compaction
8c3a180e8 Set lower-bound on dynamic level sizes
7c1c8ce5a Avoid calling fallocate with UINT64_MAX
a45e98a5b max_open_files dynamic set, follow up
6b99dbe04 fix memory alignment with logical sector size
e7ae4a3a0 Max open files mutable
60847a3b0 port: updated PhysicalCoreID()
b551104e0 support PopSavePoint for WriteBatch
498693cf3 Remove orphaned Java classes
5e2ebf2bd travis: add CMake compilation
af6fe69e4 Fix an issue of manual / auto compaction data race
6798d1f3b Revert "Delete filter before closing the table"
89833577a Delete filter before closing the table
47a09b0a8 Avoid pinning when row cache is accessed
aeaba07b2 Remove an assert that causes TSAN failure.
0b90aa951 Set VALGRIND_VER
a2b05210e Make PosixLogger::flush_pending_ atomic
da4b2070b Fix WriteBatchWithIndex address use after scope error
d616ebea2 Add GPLv2 as an alternative license.
4439b4596 Add documentation license
0ca3ead0c add GetRootDB() in DeleteFilesInRange
cdad04b05 Remove double buffering on RandomRead on Windows.
e15382c09 Disable two flaky tests
2150cc1f3 fix WritableFile buffer size in direct IO
efc361ef7 Add user stats Reset API
6616e4d62 add prefetch to PosixRandomAccessFile in buffered io
f6a27d0bc Extract statistics tests into separate file
7eddecce1 support bulk loading with universal compaction
3b4d1b7a4 add <sys/sysmacros.h> to avoid warning with glibc 2.25
e5e545a02 Reunite checkpoint and backup core logic
72c21fb3f call GetRootDB() before cast to DBImpl* in CancelAllBackgroundWork
4c9447d88 Add erase option to release cache
04d58970c AIX and Solaris Sparc Support
afff9951e Respect deprecated flag in table options
066cfbacc Adding -noprofile to CMakeLists for Windows
cb885bccf set compaction_iterator earliest_snapshot to max if no snapshot
7534ba7bd StackableDB should pass ResetStats()
c1fbf91b2 Fixing Solaris Sparc crash due to cached TLS
963eeba48 Revert how check_format_compatible.sh checkout release branches.
97005dbd5 tools/check_format_compatible.sh to cover option file loading too
8f6196788 Add cpu usage to regression benchmarks (4th attempt)
df74b775e Limit backups opened
1dd776051 Change L0 compaction score using level size
927bbab25 Revert "Add cpu usage to regression benchmarks (3rd attempt)"
8e84a388e Re-add index_per_partition but as deprecated
1553659d6 Add more recent versions to tools/check_format_compatible.sh
966ebb02f Hide event listeners from lite build
476e80be8 Add cpu usage to regression benchmarks (3rd attempt)
c49d70465 Add DB:ResetStats()
0fcdccc33 Blob storage helper methods
a6439d797 CMake: compile with -O2
e67f0adf3 enable O2 optimization for lz4
bc3973259 CMake: add support for SSE4.2
7d5f5aa97 Separate compile and link for shared library
071652734 remove warning
6e8d6f429 readahead backwards from sst end
ca96654d8 Change Build Env to gcc-5
e9e6e5324 Simplify write thread logic
6799c7e00 Pass in remote as a param to branch creation script
44fa8ece9 change use_direct_writes to use_direct_io_for_flush_and_compaction
13b50358f add space for buggy kernel warning
b6b9359ec Fix BYTES_WRITTEN accounting
13369fbd4 Update ShipIt to honor TARGETS updates
f2449ce92 Remove .deprecated_arcconfig
415be221c RocksDB Release 5.4 : Update HISTORY.md and build version.
3eab41d7c java dependencies test -s -> use test -d
a22ed4eab internal_repo_rocksdb to build Java and RocksDB LITE
9f2cc59ec sync TARGETS file
10d754696 set readahead buffer size from roundup(user_size) + 4k to roundup(use…
ba7da434a fix db_stress crash caused by buggy kernel warning
6257837d8 Add ROCKSDB_JAVA_NO_COMPRESSION flag
6a6723ee1 Move MergeOperatorPinning tests to be with other merge operator tests
6a8d5c015 Revert "Report cpu usage using time command"
8f47a9751 File level histogram should be printed per CF, not per DB
9300ef545 Fix shared lock upgrades
1f8b119ed Limit maximum memory used in the WriteBatch representation
97ec8a134 Report cpu usage using time command
20778f2f9 Adding comments to the write path
7124268a0 Reduce the number of params needed to construct DBIter
04abb2b2d FreeBSD only requires WITH_JEMALLOC, not the rest
61730186d dummy diff
360e9960f Summary: Remove .arcconfig
69a5e6461 Deprecate .arcconfig
9690653db Add a verify phase to benchmarks
dd8f9e38e Fix compilation for GCC-5
c2954f9b6 Add Travis job to build examples
72fc1e9d0 avoid non-existent O_DIRECT on OpenBSD
ff9728701 Refactor compaction picker code
9e7293902 only FALLOC_FL_PUNCH_HOLE when ftruncate is buggy
343b59d6e Move various string utility functions into string_util
1d068f606 Fix CompactRange incorrect buffer release
df6f5a377 Move memtable related files into memtable directory
107c5f6a6 CMake: more MinGW fixes
d2dce5611 Move some files under util/ to separate dirs
c50e3750d Use a human readable size for level report
ce64b8b71 Divide db/db_impl.cc
02799ad77 Revert "delete fallocate with punch_hole"
e2a7b202c Release note for partition filters
af256eb2b build db every monday
e5a1372b2 Rework test running script.
d659faad5 Level-based L0->L0 compaction
a12306fab Add a notice on gflags installation in INSTALL.md
43010a929 Revert "[rocksdb][PR] CMake: more MinGW fixes"
a30b75cdc Add buckifier script to github repo
3450ac8c1 CMake: more MinGW fixes
90cfd4645 update IterKey that can get user key and internal key explicitly
e2c6c0636 add TimedEnv
9e4453180 Refactor WriteImpl (pipeline write part 1)
6ef8c620d Move auto_roll_logger and filename out of db/
a1c469d71 Add release notes for PinnableSlice
0537f515c fix run_remote with strong quoting
72e600094 fixed misses on Centos library installation instructions
88cc81df5 auto_roll_logger_test to move away from real sleep
d25e28d58 replace sometimes-undefined uint type with unsigned int
a1d7e487b Add L0 write-amp to compaction level stats
b6d609063 CMake: support AVX2 in MinGW
bd7d13835 test remote instead run remote in regression test
c81a805fe test db existence on the remote host
5fc1e6765 add -rf when remove db in regression test
4ab4049f2 gflags has moved to GitHub
4e0065015 make all DB::Get overloads virtual
6401a8b76 Fix build with MinGW
80fe5b385 disable test: DeleteSchedulerTest.DynamicRateLimiting1
a9c86f51b backup garbage collect shared_checksum tmp files
da175f7ec exit with code 2 when there is already a db_bench running in regression test
0ee7f0403 Added missing options to RocksJava
c6d04f2ec Option to fail a request as incomplete when skipping too many internal keys
58179ec4a Cleanup of ThreadStatusUtil structures should use the DB's reference
f3607640a add ldb build to regression test
8a8c96746 Enable Fast CRC32 for Win64
f9813b853 Added SstFileWriter construtor without explicit comparator to JNI api
8d3cb4f20 Added naming of backup engine threads
67d762379 Expose the stalling information through DB::GetProperty()
0fd574926 delete fallocate with punch_hole
41fe9ad75 Hide usage of compaction_options_fifo from lite build
e7731d119 Configure index partition size
69c8d524a Fix jni library name for PowerPC Architecture
34a70859b Fix segmentation fault caused by #1961
8dee8cad9 Enable fifo compaction benchmark to db_bench
a5c8b5434 fix a header include
91b5feb37 Fix Windows Build broken by a recent commit
41ccae6d2 Add C API functions (and tests) for WriteBatchWithIndex
88bb6f6bf non_shm CI should run tests on /tmp
8888de2b1 Update .gitignore file in examples
203136e79 Fix Compilation errors when using IBM Java
f4fce4751 Fix clang compile error - [-Werror,-Wunused-lambda-capture]
a084b26a5 Blog post for releasing 5.2.1
15950fe3a Remove ASSERT_EQ(boolean, ...)
3e56c7e0c make total_log_size_ atomic
909028e21 HISTORY.md for log_size_for_flush in CreateCheckpoint()
be723a8d8 Optionally construct Post Processing Info map in MemTableInserter
e474df947 db_bench: not need to check mmap for PlainTable
8b0097b49 Readers for partition filter
9ef3627fd Allow checkpointing without flushing
17866ecc3 Allow Users to change customized ldb tools' header in help printing
a2a883318 remove deleted option from benchmark.sh
78cb19559 add checkpoint to ldb
4b04addfc updated solution if "make format" command fails
8f5bf0446 Flush triggered by DB write buffer size picks the oldest unflushed CF
6908e24b5 dynamic setting of stats_dump_period_sec through SetDBOption()
93c68b642 change regression bash file with debug mode
21d8c3179 remove LIKELY from public headers
36ad75778 INSTALL: document USE_SSE
9272e12f1 avoid ftruncate twice in buffered io
d52f334cb Break stalls when no bg work is happening
e66221add fix db_bench rate limiter callsites
dbae438a0 Replace "DEPRECATED" comment to "not supported"
995618a82 Support SstFileManager::SetDeleteRateBytesPerSecond()
e19163688 Add macros to include file name and line number during Logging
d525718a9 cleanup direct io flag in WritableFileWriter
5fa927aa9 Add Xpress and ZSTD CompressionType values to C header
11526252c Pinnableslice (2nd attempt)
e5bd8def1 update history.md for fixing the bug that skips keys
1ffbdfd9a Add a new SstFileWriter constructor without explicit comparator
ebd5639b6 Add ability to search for key prefix in sst_dump tool
e6725e8c8 Fix some bugs in MockEnv
900c62be6 fix compile for VS2015
fe1835617 release 5.3
5dae01947 Revert "Report cpu usage using time command"
f2817fb7f avoid ASSERT_EQ(false, ...);
5b11124e3 add max to histogram stats
d43adf21b Report cpu usage using time command
18fc1bc0e minor changes for rate limiter test flakiness
12ba00ea6 Reset DBIter::saved_key_ with proper user key anywhere before pass to DBIter::FindNextUserEntry
c9df05d1e Fix random access alignment
f64991537 Add Bulkoading IngestExternalFile blog post
54b434110 Builders for partition filter
97edc72d3 Add a memtable-only iterator
72202962f fix db_sst_test flakiness
5f65dc877 Expose DB::DeleteRange and WriteBath::DeleteRange in Java
58b12dfe3 Set logs as getting flushed before releasing lock, race condition fix
f8a4ea020 Move db_test and external_sst_file_test out of Travis's MAC OS run
534581a35 Fix a bug in tests in options operator=
a2f7a514d Refactoring
2a5daa06f Add stderr log level for ldb backup commands
4561275c2 fix rate limiter test flakiness
7c80a6d7d Statistic for how often rate limiter is drained
0ad5af42d Clarify VerifyBackup behavior
6fb901344 sanitize readahead when direct read enabled
f89b3893c Remove skip_table_builder_flush and default it to true
cc253982d Use more default options in db_bench
8432bcf55 Make compaction_pri settable through option string
d5b607a43 Make db_wal_test slightly faster
ba4c77bd6 Divide external_sst_file_test
e877afa08 Remove bulk loading and auto_roll_logger in rocksdb_lite
90d835507 Fix the wrong address for PREFETCH in DynamicBloom::Prefetch
08864df21 Move advanced column family options to advanced_options.h
2ca2059f6 Get unique_ptr to use delete[] for char[] in DumpMallocStats
253799c06 Add missing include for `abort()`
c6d464a9d Fixed various memory leaks and Java 8 JNI Compatibility
be3e5568b Fix unaligned reads in read cache
8ad0fcdf9 Separate small subset tests in DBTest
6c951c43c Run fewer tests in OSX
f7997f134 add direct I/O to version notes 5.2.0
3b8ba703c Fix flaky DBTestUniversalCompaction.UniversalCompactionTrivialMoveTest2
96c7e1504 Fix Java build
e0b87afc7 Black list some slow valgrind tests
e67232cff Handle failed Finish() in SST file writer
8efb5ffa2 [rocksdb][PR] Remove option min_partial_merge_operands and verify_checksums_in_comp…
1ba2804b7 Remove XFunc tests
e7d902e69 add direct_io and compaction_readahead_size in db_stress
1ef5f50e8 detect logical sector size
ed50308d2 check backup directory exists before listing children
4d7c06ced Make WriteBatchWithIndex moveble
5040414e6 Gracefully handle previous backup interrupted
f206af56f add use_direct_io() to ReadaheadRandomAccessFile
082493442 truncate patch
286a36db7 posix writablefile truncate
f0879e4c3 Page size isn't always 4k on linux
18eeb7b90 Fix interference between max_total_wal_size and db_write_buffer_size checks
1560b2f5f Temporarly return deprecated functions to fix MongoRocks build
2a0f3d0de level compaction expansion
ebc8a7998 alignment is on in ReadaheadRandomAccessFile::Read()
381fd3224 Remove timeout_hint_us from WriteOptions
fce7a6e19 Fail IngestExternalFile when bg_error_ exists
a618a16f4 New subcode for IOError to detect the ESTALE errno
7ab005183 Remove deprecated DB::AddFile and DB::CompactRange
401667366 Adding Dlang to the list
756c5924e Allow adding external v1 sst file with no global seqno support
aa0298fa9 Remove trailing whitespace from examples Makefile
db2b4eb50 avoid direct io in rocksdb_lite
43e9f01c2 Fix repair_test on ROCKSDB_LITE
7106a994f Use monotonic time points in write_controller.cc and rate_limiter.cc
c2247dc1c Make DBImpl::has_unpersisted_data_ atomic
eb912a927 Remove disableDataSync option
0227c16d6 Update static library versions and add checksums
b3aae4d07 Add repair_test to make check
421ce7c2b Add support for JNI Library on Linux on PowerPC.
9afa20cf2 Increase build version and HISTORY.md for releasing 5.2
a5adda064 Fix repair issues
b48e4778b Consolidate file cutting logic in compaction loop
ac2a77a74 Announce the experimetnal two-level index feature in HISTORY.md
c4a37dcb4 Print the missed last layer in cfstats
a12818afc Blog post for 5.1.2 release
3b4ac8076 Clarify ldb column family argument
d70ce7ee0 Move db_bench flags out of unnamed namespace
186c7eedb Remove incorrect statistics warning
53bb01516 [rocksdb][PR] compaction_style and compaction_pri should output their value as a st…
69d5262c8 Two-level Indexes
0a4cdde50 Windows thread
1aaa898cf Adding GetApproximateMemTableStats method
9fc23c55f Use gcc-4.9-glibc-2.20-fb python in precommit_checker
b797e4215 Dump compression dictionary meta-block
036d668b1 Fix wrong result in data race case related to Get()
574b543f8 Rename merger.h -> merging_iterator.h
add8b50cc Move ThreadLocal implementation into .cc
71d2496af Fix arc setting for Facebook internal tools
f289d9f4a Fix OSX build break after the fallocate change
4a3e7d320 Change the default of delayed slowdown value to 16MB/s
0513e21f9 RangeSync() should work with ROCKSDB_FALLOCATE_PRESENT not set
8b369ae5b Cleaner default options using C++11 in-class init
ec79a7b53 Dedup code in option.cc and db_options.cc
b96372dea improving the C wrapper
04c4ec41d Change corruption_test to use 4 bits.
2d75cd40d NewLRUCache() to pick number of shard bits based on capacity if not given
f25f1ec60 Add test DBTest2.GetRaceFlush which can expose a data race bug
37d4a79e9 Deserialize custom Statistics object in db_bench
3b35134e4 Avoid cache lookups for range deletion meta-block
94a0c32e7 Fix LRU Ref() for handles with external references only
17c118060 Generalize Env registration framework
07dddd5f7 EnvPosixTestWithParam should wait for all threads to finish
5dad9d6d2 Avoid logs_ operation out of DB mutex
a7b13919b Fix CompactFiles() bug when used with CompactionFilter using SuperVersion
616a1464e Fix DeleteRange including sentinels in output files
c918c4b76 Update USERS.md add user Pika
03ca2ac8a Remove function from DBImpl that are not used anywhere
b0029bc7f Test merge op covered by range deletion in memtable
d438e1ec1 Test range deletion block outlives table reader
fba726e55 Version librocksdb.so
9da4d542f Range deletions unsupported in tailing iterator
f2b4939da fixed typo
973f1b78f memtable: delete merge value for range deleteion
aebfd1703 fix non-portable behavior in encoder
753ff84a3 Fix get approx size
d7ea44f2f Fixup a couple of builds errors on Linux.
537da370d c: allow set savepoint to writebatch
af6ec4d78 fix batchresult handle leak
e29bb934f Zlib 1.2.8 is no longer available, switched to 1.2.10
5ac97314e Fix std::out_of_range when DBOptions::keep_log_file_num is zero
4e35ffdfa cmake: check -momit-leaf-frame-pointer before using it
3c0852d1d Make ingest external file backward compatible
0e8dfd606 Fix OptimizeForPointLookup()
e840213d6 Change DB::GetApproximateSizes for more flexibility needed for MyRocks
9239103cd Flush job should release reference current version if sync log failed
da54d36a9 Disable IngestExternalFile in ReadOnly mode
5cf176ca1 Fix for 2PC causing WAL to grow too large
4a73bb0b4 Split travis jobs
c70d3c7ad Enable DBTest.GroupCommit as it runs in a reasonlable time now.
602c13a96 Remove fadvise with direct IO read
f9d18e22d Fix DeleteRange file boundary correctness issue with max_compaction_bytes
3ce091fd7 Add KEEP_DB env var option
77b480662 Fix 2PC with concurrent memtable insert
e8a096000 util/thread_local.h: silence a clang-build warning
324a0f988 Follow up for DirectIO refactor
bc5d7b702 travis: For linux, do all tests under gcc
3e6899d11 change UseDirectIO() to use_direct_io()
d4e07a845 fix warning of unused direct io helper functions
dc2584eea direct reads refactor
d18dd2c41 Abort compactions more reliably when closing DB
62384ebe9 Guarding extra fallocate call with TRAVIS because its not working pro…
9f246298e Performance: Iterate vector by reference
fe395fb63 Allow incrementing refcount on cache handles
2172b660e Fix build on FreeBSD
3c233ca4e Fix Windows environment issues
763173456 Fix the error in ColumnFamiliesTest
7a02ad070 Update travis to ubuntu trusty
60c509ff1 Fix valgrind failure in test CurrentFileModifiedWhileCheckpointing2PC
d0ba8ec8f Revert "PinnableSlice"
54d94e9c2 PinnableSlice
e04480fae Fix MS warnings. Use ROCKSDB_Prsz for size_t.
c081f7215 5.0.1 release blog post
ac73d7558 Add GetSupportedCompressions() convenience function
b104b8781 Maintain position in range deletions map
640d72480 Update db_bench and sst_dump to test with block cache mid-point inser?
653ac1f9c C API: support total_order_mode
85ac1a320 Fix rocksdb::Status::getState
76711b6e7 Make ExternalSSTFileTest::CompactionDeadlock more deterministic
c963460db Fix tests under GCC_481
33c86d677 Fix backupable db test
e425ec116 utilities/backupable: backup should limit the copy size of wal.
0712d541d Delegate Cleanables
d58ef52ba Allow SstFileWriter to Fadvise the file away from page cache
17a4b75cc Always fsync the file after file copying
2fb70dc79 examples: Add options_file_example to target all
a738af8f8 db/pinned_iterators_manager.h: bugfix
906523d98 Add description to the 2PC checkpooint bug in HISTORY.md
438f22bc5 Fix bug of Checkpoint loses recent transactions with 2PC
335981d47 Fix the directory path for RocksDB repo
548b62805 Enable conditionally using adaptive mutexes
4e07b08ef include/rocksdb/utilities/env_librados: fix typo
ab48c165a Print cache options to info log
972f96b3f direct io write support
989e644ed Remove sst_file_manager option from LITE
1beef6569 Fix c_test
3d692822f persistent_cache: fix two timer
046099c9b The array is malloced by backtrace_symbols(), and must be freed
6ff2c8d7f Remove gflags as travis build dependency
3cd9ed1c3 Show sandcastle URL in phabricator
50e305de9 Collapse range deletions
5d1457dbb Dump persistent cache options
7bd725e96 db_bench: introduce --benchmark_read_rate_limit
296691847 Update Netflix section of USERS.md
342370f1d Simplify MemTable::Update
1a136c1f1 Expose file size
fbff4628a Reduce compaction iterator status checks
bd6cf7b51 WritableFileWriter: default buffer size equal min(64k,options.writabl?
fc0c6fd98 "make format" format diff since last commit from master
816c1e30c gcc-7 requires include <functional> for std::function
c27073586 Iterator should be in corrupted status if merge operator return false
a8bf4d63f Make format shows wrong curl command for retrieving clang-format-diff.py
8f5d24ae6 C API: support get usage and pinned_usage for cache
0ab6fc167 Gcc-7 buffer size insufficient
b7239bf7e Gcc 7 fallthrough
477b6ea57 std::remove_if requires <algorithm>
83f9a6fd2 Fail BackupEngine::Open upon meta-file read error
a79eae4b0 Add pcache documentation images
d71e728c7 Print user collected properties in sst_dump
7004a6f7b Add missing copyright header
3cdfaeca3 Fixes for MSVC compilation
e097222e6 util/logging.cc: buffer of insufficient size (gcc-7 -Werror=format-length)
cfc34d7c4 Missing break in case in DBTestBase::CurrentOptions
bfbcec233 Gcc 7 error expansion to defined
6653e32ac build: make it easier to pass PORTABLE
67adc937b intentional fallthough (prevents gcc-7/clang-4 error)
1a146f89c break Flush wait for dropped CF
c3e5ee715 util/histogram.cc: HistogramStat::toString buffer insufficient
5334d8b44 table/block_based_table_builder.cc: intentional fallthrough - comment to match gcc pattern
36d42e65d Disable test to unblock travis build
b57dd9262 C API: support writebatch delete range
2ba59b5a1 Disallow ingesting files into dropped CFs
1f6f7e3e8 cast to signed char in ldb_cmd_test for ppc64le
243975d5d More accurate error status for BackupEngine::Open
f0c509e2c Return finer-granularity status from Env::GetChildren*
dc64f46b1 Add db_bench option for stderr logging
2cabdb8f4 Increase buffer size
4a17b47bb Remove unnecessary header include
8c2b921fd Fixed a crash in debug build in flush_job.cc
20ce081fa Fix issue where IngestExternalFile insert blocks in block cache with g_seqno=0
5241e0dbf fix db_bench argument type
c04f6a0b4 Specify shell in makefile
45c7ce137 CompactRangeOptions C API
2c2ba6824 db_stress support for range deletions
b821984d3 DeleteRange read path end-to-end tests
2f4fc539c Compaction::IsTrivialMove relaxing
1dce75b2e Update USERS.md
304b3c706 Update USERS.md
fa50fffaf Option to expand range tombstones in db_bench
c26a4d8e8 Fix compile error in trasaction_lock_mgr.cc
ed8fbdb56 Add EventListener::OnExternalFileIngested() event
2005c88a7 Implement non-exclusive locks
0b0f23572 Mention IngestExternalFile changes in HISTORY.md
23db48e8d Update HISTORY.md for 5.0 branch
beb36d9c1 Fixed CompactionFilter::Decision::kRemoveAndSkipUntil
67f37cf19 Allow user to specify a CF for SST files generated by SstFileWriter
9053fe2a5 Made delete_obsolete_files_period_micros option dynamic
edde954e7 fix clang build
56281f3a9 Add memtable_insert_with_hint_prefix_size option to db_bench
4a21b1402 Cache heap::downheap() root comparison (optimize heap cmp call)
e39d08087 Fix travis (compile for clang < 3.9)
3f407b065 Kill flashcache code in RocksDB
b77007df8 Bug: paralle_group status updated in WriteThread::CompleteParallelWorker
247d0979a Support for range skips in compaction filter
96fcefbf1 c api: expose option for dynamic level size target
00197cff3 Add C API to set base_backgroud_compactions
5b219eccb deleterange end-to-end test improvements for lite/robustness
aad119176 pass rocksdb oncall to mysql_mtr_filter otherwise tasks get created w…
e33352899 DeleteRange write path end-to-end tests
7784980fc Fix mis-reporting of compaction read bytes to the base level
3c6b49ed6 Fix implicit conversion between int64_t to int
b3b875657 Remove unused assignment in db/db_iter.cc
4f6e89b1d Fix range deletion covering key in same SST file
a2bf265a3 Avoid intentional overflow in GetL0ThresholdSpeedupCompaction
52fd1ff2c disable UBSAN for functions with intentional -ve shift / overflow
1886c435b Fix CompactionJob::Install division by zero
63c30de80 fix options_test ubsan
13e66a8f5 Fix compaction_job.cc division by zero
01eabf737 Fix double-counted deletion stat
7ffb10fc1 DeleteRange compaction statistics
236d4c67e Less linear search in DBIter::Seek() when keys are overwritten a lot
cd7c4143d Improve Write Stalling System
dfb6fe675 Unified InlineSkipList::Insert algorithm with hinting
3068870cc Making persistent cache more resilient to filesystem failures
734e4acaf Eliminate redundant cache lookup with range deletion
182b940e7 Add WriteOptions.no_slowdown
4118e1333 Persistent Cache: Expose stats to user via public API
f2a8f92a1 rocks_lua_compaction_filter: add unused attribute to a variable
4444256ab Remove use of deprecated LZ4 function
548d7fb26 Fix fd leak when using direct IOs
fd43ee09d Range deletion microoptimizations
23a18ca5a Reword support a little bit to more clear and concise
481856ac4 Update support to separate code issues with general questions
a0deec960 Fix deadlock when calling getMergedHistogram
fe349db57 Remove Arena in RangeDelAggregator
e63350e72 Use more efficient hash map for deadlock detection
a13bde39e Skip ldb test in Travis
73843aa63 Direct I/O Reads Handle the last sector correctly.
9d60151b0 Implement PositionedAppend for PosixWritableFile
3f6221521 Lazily initialize RangeDelAggregator's map and pinning manager
41e77b839 cmake: s/STEQUAL/STREQUAL/
c1038d283 Release RocksDB 5.0
635a7bd1a refactor TableCache Get/NewIterator for single exit points
f39452e81 Fix heap use after free ASAN/Valgrind
a4eb7387b Allow plain table to store index on file with bloom filter disabled
36e4762ce Remove Ticker::SEQUENCE_NUMBER
86eb2b9ad Fix src.mk
0765babe1 Remove LATEST_BACKUP file
647eafdc2 Introduce Lua Extension: RocksLuaCompactionFilter
760ef68a6 fix deleterange asan issue
327085b7b fix valgrind
715591bba Ask travis to use JDK 7
972e3ff29 Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default
420bdb42e option_change_migration_test: force full compaction when needed
1543d5d92 Report memory usage by memtable insert hints map.
018bb2ebf DeleteRange support for db_bench
dc51bd716 CMakeLists.txt: FreeBSD has jemalloc as default malloc
48e8baebc Decouple data iterator and range deletion iterator in TableCache
4b0aa3c4c Fix failed compaction_filter_example and add it into make all
53b693f5f ldb support for range delete
661e4c926 DeleteRange unsupported in non-block-based tables
489d14280 DeleteRange interface
eba99c28e Fix min_write_buffer_number_to_merge = 0 bug
2ef92fea5 Remove all instances of relative_url until GitHub pages problem is fixed.
91300d01f Dynamic max_total_wal_size option
ec2f64794 Consider subcompaction boundaries when updating file boundaries for range deletion
800e51553 Fix CSS issues again
b952c898b Parallize persistent_cache_test and transaction_test
3b192f618 Handle full final subcompaction output file with range deletions
6c5795200 Make range deletion inclusive-exclusive
425210cc4 CSS issues are arising on the Github Pages side. Temp fix.
1ea79a78c Optimize sequential insert into memtable - Part 1: Interface
df5eeb85c Optimize sequential insert into memtable - Part 2: Implementation
5ed650857 Fix SstFileWriter destructor
adb665e0b Allowed delayed_write_rate option to be dynamically set.
307a4e80c sst_dump support for range deletion
361010d44 Exporting compaction stats in the form of a map
672300f47 Use relative Urls for stylesheets
b39b2ee12 do not call get() in recovery mode
1ca5f6d13 Fix 2PC Recovery SeqId Miscount
e095d0cbc Rocksdb contruns to new Sandcastle API
14c0380e7 Convenience option to parse an internal key on command line
c90fef88b fix open failure with empty wal
4e20c5da2 Store internal keys in TombstoneMap
a9fb346e4 Fix RocksDB Lite build failure in c_test.cc
d133b08f6 Use correct sequence number when creating memtable
144cdb8f1 16384 as e.g .value for compression_max_dict_bytes
9bd191d2f Fix deadlock between (WriterThread/Compaction/IngestExternalFile)
a9fae0a9d CSS problems again :(
193221e0a Fix Forward Iterator Seek()/SeekToFirst()
e48f3f8b9 remove tabs and duplicate #include in c api
85bd8f518 Minor fix to GFLAGS usage in persistent cache
a7875272d c: support seek_for_prev
0f17f9279 Make the header links a bit more flexible
cf19f559d single quotes in feed
2dc019e09 Fix header links
f1aedda06 More Jekyll 3.3 fixes
c54cdc378 More Jekyll 3.3 updates
2bcaf8246 Update product and feature template for Jekyll 3.3
24bceb096 Java API - Implement GetFromBatch and GetFromBatchAndDB in WBWI
815f54afa Insert range deletion meta-block into block cache
9e7cf3469 DeleteRange user iterator support
5c5d01ae7 Fix wrong comment (Maximum supported block size)
f998c9790 DeleteRange Get support
879f36636 Add C api for RateLimiter
557034f36 Remove all instances of baseurl
437942e48 Add avoid_flush_during_shutdown DB option
2b16d664c Change max_bytes_for_level_multiplier to double
16fb04434 expose IngestExternalFile to c abi
ce22ea99a Fix casts for MSVC
196af035c Introduce FAIL_ON_WARNINGS CMake variable (default ON)
40a2e406f DeleteRange flush support
d5555d95a Fix MSVC compile error in 32 bit compilation
da61f348d Print compression and Fast CRC support info as Header level
f9eb56791 db_bench: --dump_malloc_stats takes no effect
6a4faee5c fix freebsd build include path err and so & jar file name
c90c48d3c Show More DB Stats in info logs
1b295ac8a DBTest.GetThreadStatus: Wait for test results for longer
25f5742f0 Update documentation to point at gcc 4.8
b50a81a2b Add a test for tailing_iterator
04751d534 L0 compression should follow options.compression_per_level if not empty
2946cadc4 Improve RangeDelAggregator documentation
0a9fd05c2 Update Vagrant file (test internal phabricator workflow)
fcd1e0bf6 Make rocksdb work with internal repo
0aab5e55f FreeBSD: malloc_usable_size is in <malloc_np.h> (#1428)
9c0bb7f17 cmake: drop "-march=native" from CXX_FLAGS (#1429)
eeb27e1bb Add handy option to turn on direct I/O in db_bench (#1424)
c6168d13a removed some declarations from c.h which resulted in undefined symbols (#1407)
bc429de49 revert fractional cascading in farward iterator
b9bc7a2aa Use skiplist rep for range tombstone memtable
60a2bbba9 Makefile: generate util/build_version.cc from .in file (#1384)
9ee84067f Disable DBTest.RepeatedWritesToSameKey (#1420)
f41df3045 OptionChangeMigration() to support FIFO compaction
2e8004e60 Changing the legocastle run to use valgrind_test instead of _check
9de2f7521 revert Prev() in MergingIterator to use previous code in non-prefix-seek mode
24495186d DBSSTTest.RateLimitedDelete: not to use real clock
1168cb810 Fix a bug that may cause a deleted row to appear again
99c052a34 Fix integer overflow in GetL0ThresholdSpeedupCompaction (#1378)
f83cd64c0 Fix a bug that mistakenly disable regression_test.sh to update commit (#1415)
0e926b84f Passing DISABLE_JEMALLOC=1 to valgrind_check if run locally
4dfaa6610 Make IsDeadlockDetect() virtual member of Transaction
59a7c0337 Change ioptions to store user_comparator, fix bug
869ae5d78 Support IngestExternalFile (remove AddFile restrictions)
1d9dbef64 Restrict running condition of UniversalCompactionTrivialMoveTest2
4edd39fda Implement deadlock detection
48fd619a4 Minor fixes to RocksJava Native Library initialization (#1287)
48e4e842b Disable auto compactions in memory_test and re-enable the test (#1408)
fb2e41294 column_family_test: disable some tests in LITE
5af651db2 fix data race in compact_files_test
a0ba0aa87 Fix uninitialized variable gcc error for MyRocks
b88f8e87c Support SST files with Global sequence numbers [reland]
08616b493 [db_bench] add filldeterministic (Universal+level compaction)
52c9808c3 not split file in compaciton on level 0
5e0d6b4cc fix db_stress assertion failure
ab5399837 Bump RocksDB version to 4.13 (#1405)
b4d07123c SamePrefixTest.InDomainTest to clear the test directory before testing
aa09d0338 Avoid calling GetDBOptions() inside GetFromBatchAndDB()
6fbe96baf Compaction Support for Range Deletion
257de78d9 Remove "-Xcheck:jni" from Java tests (#1402)
d88dff4ef add seeforprev in history
5027dd17a Fix a minor bug in the ldb tool that was not selecting the specified (#1399)
fea6fdd67 Fix @see in two Java functions (#1396)
b1031d6c1 Remove function local statics that interfere with memory pooling (#1392)
f47054015 Handle WAL deletion when using avoid_flush_during_recovery
e29d3b67c Make max_background_compactions and base_background_compactions dynamic changeable
21e8daced fix assertion failure in Prev()
b9311aa65 Implement WinRandomRW file and improve code reuse (#1388)
a249a0b75 check_format_compatible.sh to use some branch which allows to run with GCC 4.8 (#1393)
040328a30 Remove an assertion for single-delete in MergeHelper::MergeUntil
8cbe3e10c Relax the acceptable bias RateLimiterTest::Rate test be 25%
f26a139d8 Log successful AddFile
5691a1d8a Fix compaction conflict with running compaction
017de666c fixup commit
1b7af5fb1 Redo handling of recycled logs in full purge
27bfe327b Editorial change to README.md
89cc404de A bit of doc restructuring
9e7fda829 Fix arcanist
2e4b5cab0 Add missing RateLimiter class to the Windows build (#1382)
ce4963fdf [doc] Document that Visual Studio 2015+ is now required for Windows builds (#1389)
e48927098 Fix scoped arena iterator (#1387)
f8d8cf53f Fix log_write_bench -bytes_per_sync option. (#1375)
02b3e3985 Make txn->GetState() const
447f17127 new Prev() prefix support using SeekForPrev()
991b585ee More block cache tickers
d6ae6dec6 Add Statistics::getAndResetTickerCount().
aea3ce4c8 Avoid string CONCAT which is not supported in cmake 2.6 (#1383)
2ad68b971 Support running consistency checks in release mode
67501cfc9 Fix -ve std::string::resize
04b02dd12 Testing asset links after config change
8c55bb87c Make Lock Info test multiple column families
d06232897 Revert "Support SST files with Global sequence numbers"
5cd28833a [RocksJava] Adjusted RateLimiter to 3.10.0 (#1368)
37737c3a6 Expose Transaction State Publicly
2c1f95291 Add facility to write only a portion of WriteBatch to WAL
043cb62d6 Fix record_size in log_write_bench, swap args to std::string::assign. (#1373)
4985f60fc env_mirror: fix a few leaks (#1363)
5aded67dd update of c.h (#1371)
912aec13c "Recent Posts" -> "All Posts"
7cbb298db Make sure that when contribtuing we call out creating appropriate directories
a06ad4711 Add top level doc information to CONTRIBUTING.md
3fdd5b971 A little more generic CONTRIBUTING.md
ed4fc31db Add link to CONTRIBUTING.md to main docs README.md
e4922e181 Forgot to truncate one blog post
6d8cd7ede Add CONTRIBUTING.md for rocksdb.org contribution guidance
bd55e5a1e Fix some formatting of compaction blog post
0f60358b0 CRLF -> LF mod (including removing trailing whitespace for those files)
b90e29c90 Truncate posts on the main /blog/ page
0d7acadaf Add author fields to blog posts
01be44181 Add GitHub link to the landing page header
9d6c96138 Fix Mac build
ab01da543 Support SST files with Global sequence numbers
d346ba246 Minor fixes around Windows 64 Java Artifacts (#1366)
e91b4d0cf Add factory method for creating persistent cache that is accessible from public
be1f1092c Expose transaction id, lock state information and transaction wait information
6009c473c Store range tombstones in memtable
3c21c64c7 Use size hint for HashMap in multiGet. Similar to https://github.com/facebook/rocksdb/pull/1344 (#1367)
13f7a01f6 Fixing JNI release build for gcc (#975)
7260662b3 Add Java API for SstFileWriter
26388247a delete unused variable for PrevInterval()
87dfc1d23 Fix conflict between AddFile() and CompactRange()
eb44ed655 Update 2016-09-28-rocksdb-4-11-2-released.markdown
e4437610d Update 2016-09-28-rocksdb-4-11-2-released.markdown
501f05108 Update 2016-09-28-rocksdb-4-11-2-released.markdown
dec9009f8 Update 2016-09-28-rocksdb-4-11-2-released.markdown
4ed69dd0b Create 2016-09-28-rocksdb-4-11-2-released.markdown
21f4bb5a8 cmake support for linux and osx (#1358)
4defe306f fix typo in comments (#1360)
f517d9dd0 Add SeekForPrev() to Iterator
eb3894cf4 Recompute compaction score on SetOptions (#1346)
5c64fb67d Fix AddFile() conflict with compaction output [WaitForAddFile()]
9e9f5a0b9 Fix CompactFilesTest.ObsoleteFiles timeout (#1353)
c2a62a4cb not cut compaction output when compact to level 0
9ed928e7a Split DBOptions into ImmutableDBOptions and MutableDBOptions
4bc8c88e6 Recover same sequence id from WAL (#1350)
0a1bd9c50 add cfh deletion started listener
da5a9a65c Fix mac build
d45eb6c6d Fix typo (#1349)
abc0ae462 Add AddFile() InternalStats for Total files/L0 files/total keys ingested
715256338 forbid merge during recovery
5735b3dc2 Fix compiling under -Werror=missing-field-initializers
654ed9a28 loose the assertion condition of rate_limiter_test
e4d3f5d9b Fix DBImpl::GetWalPreallocateBlockSize Mac build error
7afbb7420 solve the problem of table_factory_to_write_=nullptr (#1342)
d78a4401b DBImpl::GetWalPreallocateBlockSize() should return size_t
42ac9c5f1 Retry getting arcanist token on failure
b666f8544 Consider more factors when determining preallocation size of WAL files
4c3f4496b Add TableBuilderOptions::level and relevant changes (#1335)
3edb9461b Avoid hard-coded sleep in EnvPosixTestWithParam.TwoPools
0a88f38b7 Remove ColumnFamilyData::options()
41a9070f8 Fix java makefile dependencies
8d9bf5c49 Fix DBOptionsTest.GetLatestOptions
40cfa3e02 Fix DBWALTest.RecoveryWithLogDataForSomeCFs with mac
06b4785fe Fix recovery for WALs without data for all CFs
d7242ff4d Fix GetSortedWalFiles when log recycling enabled
17f76fc56 DB::GetOptions() reflect dynamic changed options
215d12826 Fix typo (#903)
a958c2643 Rename jvalue to jval in rocksjni
0a165bd7d Have Facebook link point to RocksDB on FB
3639f3288 Fix bug in UnScSigned-off-by: xh931076284 <931076284@qq.com> (#1336)
8e061f974 Refactor GetMutableOptionsFromStrings
81747f1be Refactor MutableCFOptions
ba65c816b Support POSIX RandomRWFile
1d980a8e3 Create CNAME
2adab1dde Add API links to the header bar
a182b2981 Preserve blog comments in markdown
f54de9230 Adding Dgraph to list of Users (#1291)
9e4aa798c Summary: (#1313)
a10e8a056 Fix C api memtable rep bugs. (#1328)
eb1d4d53c Release RocksDB 4.12
22d88e24d Allow an offset as well as a length to be specified for byte[] operations in RocksJava JNI (#1264)
b06b19136 add C api for set wal_recovery_mode (#1327)
1cca09129 Temporarily revert Prev() prefix support
de28a2553 Update HISTORY.md for thread-local stats
0fcb6dbed Remove extraneous function prototypes from c.h (#1326)
52ee07b02 Move AddFile() tests to external_sst_file_test.cc
66a91e260 Add NoSpace subcode to IOError (#1320)
67036c040 Fix Flaky ColumnFamilyTest.FlushCloseWALFiles
0e2da497c fix typo in option.h's comment (#1321)
6d61358a0 Add real Google Analytics ID
2d9d36ea4 Have "Edit on GitHub" point to master instead of gh-pages
937751898 Update landing page content
1ec75ee76 Add redirects from old blog posts link to new format
607628d34 Support ZSTD with finalized format
ce1be2ce3 Fix build error on Windows (AppVeyor) (#1315)
f7669b40b Fix Windows Build
22696b088 Fix uninitlized CompactionJob::SubcompactionState::current_output_file_size
c1865e0f7 Trigger more tests per diff
a88677d2c Remove ImmutableCFOptions from public API
80c75593e Fix data race in AddFile() with multiple files + custom comparator bug
5051755e3 Fix db_bench memory use after free (detected by clang_analyze)
4fd08f4b8 Ensure Correct Behavior of StatsLevel kExceptDetailedTimers and kExceptTimeForMutex (#1308)
e14fbaae2 Add FAQ based on the front page of the current rocksdb.org
3c2262400 Migrate the RocksDB Worpdress blog over to Jekyll
ee0e2201e Transfer the current Getting Started contents to GitHub Pages
5a0e9a4cf Initial Landing Page
9447a8540 Remove the `doc` directory
32149059f Merge options source_compaction_factor, max_grandparent_overlap_bytes and expanded_compaction_factor into max_compaction_bytes
4590b53a4 add stats to Cache::LookUp()
85bb30825 Expose Utility function StringToMap() (#1306)
8ce1b8440 Fix Travis on Mac
380e651af Fix Mac build failure (#1309)
1613fa949 Thread-specific histogram statistics
6a14d55bd add prefix_seek_mode to db_iter_test
de47e2bd4 Fix ClockCache memory leak
f099af4c7 Fix travis
db74b1a21 fix bug in merge_iterator when data race happens
b18f9c9ea add nullptr check to internal_prefix_transform
4e395e875 Update docs README.md
2482d5fb4 support Prev() in prefix seek mode
7541c7a79 Fix cache_test valgrind_check failure
c8513cde0 Update the download location of Snappy (#1304)
b49b92cf2 Introduce Read amplification bitmap (read amp statistics)
c7004840d store prefix_extractor_name in table
4ad928e17 add comment to SimCache to estimate actual capacity
e9b2af87f Expose ThreadPool under include/rocksdb/threadpool.h
23a057007 Document memtable flush behavior in CancelAllBackgroundWork()
dade61ac2 Mitigate regression bug of options.max_successive_merges hit during DB Recovery
cce702a6e [db_bench] Support single benchmark arguments (Repeat for X times, Warm up for X times), Support CombinedStats (AVG / MEDIAN)
3586901f8 cat tests logs sorted by exit code
b2ce59537 Persist data during user initiated shutdown
4b3438d2d Fix parallel valgrind (valgrind_check)
a081f798b Relax consistency for thread-local ticker stats
b10d65c2a Update and slightly clarify instructions in build_detect_platform (#1301)
f85f99bf6 Fix the Windows build of RocksDB Java. Similar to https://github.com/facebook/rocksdb/issues/1220 (#1284)
7b8109517 Fix a crash when compaction fails to open a file
7c9586837 Thread-specific ticker statistics
ea9e0757f Add initial GitHub pages infra for RocksDB documentation move and update. (#1294)
2a9c97108 [Flaky Test] Disable DBPropertiesTest.GetProperty
d76ddf327 Disable ClockCache db_crashtest
cec2c6436 fix data race in NewIndexIterator() in block_based_table_reader.cc
badbff65b Not insert into block cache if cache is full and not holding handle
4a16c32ec Option to cache index/filter blocks with priority
99c4af716 Make ClockCache available with TSAN build
f57bc1d03 Fix lambda expression for clang/windows
5440675c3 Fix lambda capture expression for windows
6584cec8f Fold function for thread-local data
817eeb29b Add singleDelete to RocksJava (#1275)
ffdf6eee1 Add Status to RocksDBException so that meaningful function result Status from the C++ API isn't lost (#1273)
ecf900386 Fix bug in printing values for block-based table
72f8cc703 LRU cache mid-point insertion
6a17b07ca Add TablePropertiesCollector support in SstFileWriter
78837f5d6 TableBuilder / TableReader support for range deletion
4cc37f59e Introduce ClockCache
ff17a2abf Adding TBB as dependency.
49d88be02 c abi: allow compaction filter ignore snapshot (#1268)
0b63f51fb fixes 1215: execute_process(COMMAND mkdir ${DIR}) fails to create a directory with cmake on Windows (#1219)
3981345be Small nits (#1280)
2a2ebb6f5 Move LRUCache structs to lru_cache.h header
2fc2fd92a Single Delete Mismatch and Fallthrough statistics
3771e3797 WriteBatch support for range deletion
236756f2c Make SyncPoint return immediately when disabled
64a0082c6 Fix DBSSTest::AddExternalSstFileSkipSnapshot valgrind fail
dd7a748cf Fix java build
4fe12baa6 Make db_bench less space for --stats_per_interval
6525ce4ca Compaction stats printing: "batch" => "commit group"
a117891b4 Fixed typo (#1279)
b248e98cf Fix a destruction order issue in ThreadStatusUpdater
deda159b5 Added min/max/avg data block size output to sst_dump
e408e98c8 add Name() to Cache
a297643f2 Fix valgrind memory leak
d11c09d9e Eliminate memcpy from ForwardIterator
d36755502 Added further Java API options for controlling concurrent writes
ebdfe34cc Exposed further Java API options for controlling compaction
d1be59463 Improve documentation of SliceTransform.
6056d6317 Improve comment and bug fix for GetOptionsFromMap functions in convenience.h
76a67cf74 support stackableDB as the baseDB of transactionDB
67c1ae883 Travis build break fix
b693ba68b Minor PinnedIteratorsManager Refactoring
db3dfb164 Fixes for arcanist config (#1271)
87c91bd87 Persistent Read Cache (8) Benchmark tooling
2914de64e add sim_cache stats to Statistics
8b79422b5 [Proof-Of-Concept] RocksDB Blob Storage with a blob log file.
4beffe001 Fix test data race in two FaultInjectionTest tests
821bcb0b3 util/arena.cc: FreeBSD: More portable use of mmap(MAP_ANON) (#1254)
5370f44a8 Increase RocksDB version
56dd03411 read_options.background_purge_on_iterator_cleanup to cover forward iterator and log file closing too.
ccecf3f4f UniversalCompaction should ignore sorted runs being compacted (when compacting for file num)
1b0069ce2 Remove non-gtest from parallelized tests
638c49f24 Change HISTORY.md for release 4.11
6b8e9c68b fix vs generator (#1269)
c38b075e7 Update HISTORY.md
8f399e3fe Update HISTORY.md
98d0b78ea Added check_snapshot option in the DB's AddFile function (#1261)
9fd68b7fb set travis open file descriptor limit
59ddb5059 Fix travis build break
f4d986364 Added SetOptions support to RocksJava (#1243)
7882cb977 Make DBOptionsTest::EnableAutoCompactionAndTriggerStall less falky
44f5cc57a Add time series database (resubmitted)
7c4615cf1 A utility function to help users migrate DB after options change
5bb0a7f73 Update appveyor.yml
86396cc18 Update appveyor.yml
c1db098dc Update appveyor.yml
7da2eaf0d Update appveyor.yml
34723b4c4 Cleanup unused variable pending_fsync_.
7cc0dbd66 cat all logs in sandcastle output
9253767a6 Correct geHistogramData() -> getHistogramData() (#1257)
f35b16f24 db_bench add an option of --base_background_compactions
c3a4bea5d Fix flaky test `ObsoleteFiles`
8234faabf Fix failed test
4990c0d1a Remove deprecated LEVELDB_PLATFORM_POSIX
7323e4c8a Fix clang on macOS
ee027fc19 Ignore write stall triggers when auto-compaction is disabled
e4609a749 Fix Windows build issues (#1253)
2306167d3 Fix clang build failure and refactor unit test
343304e1d Use StopWatch to do statistic job in db_impl_add_file.cc
cdc4eb689 Add a GetComparator() function to the ColumnFamilyHandle base class so that the user's comparator can be retrieved.
712dd27e6 Build break fixes
0155c73de Fix parallel tests `make check -j`
c49ea68c5 Fix to enable running CI jobs locally
726c2f7e5 Build break fix
d51dc96a7 Experiments on column-aware encodings
c116b4780 Persistent Read Cache (part 6) Block Cache Tier Implementation
64046e581 Write a benchmark to emulate time series data
9ae92f50b More granular steps in the Makefile, can help with running all or single Java tests (and with ASAN build - https://github.com/facebook/rocksdb/wiki/JNI-Debugging) (#1237)
7c01d6534 [Fix Java] Remove duplicate cases in LoggerJniCallback::Logv
8796934af Added missing Java ReadOptions settings (#1109)
5e2c79658 Make DBTest.CompressionStatsTest more deterministic
557748ff7 Fix db_stress failure (pass merge_operator even if not used)
811ee2111 Bugfix to ensure that logging can be achieved from threads that are not known to the JVM (#1106)
afad5bd1c Simplify thread-local static initialization
6920cde89 Remove an extra apostrophe
e72ea485e add InDomain regression test
9c8ac144b Avoid duplicate task creation for RocksDB contruns
d4c45428a db_stress shouldn't assert file size 0 if file creation fails
d3bfd3397 Testing out parallel sandcastle changes
7efd9c25c Increse timeout in some tests
50b8d29b9 fixes 1230: Error:string sub-command REGEX, mode REPLACE needs at least 6 arguments total to command (#1231)
5c858ddd2 fix errata in libnuma test (#1244)
e5b5f12b8 Change options memtable_prefix_bloom_huge_page_tlb_size => memtable_huge_page_size and cover huge page to memtable too
0ce258f9b Compaction picker to expand output level files for keys cross files' boundary too.
ac0d93b08 fixes 1217: rocksdbjni javac and javah execute_processes fail on windows (#1218)
1ae46094d Appveyor badge to show master branch
8745f013f [Fix java build] Stop using non standard std::make_unique
e12270dfe fix previous typo
bbd6a5a18 ldb restore subcommand
9498069fc Run error-filtering script on diff-triggered tests
f8061a237 Fix Statistics TickersNameMap miss match with Tickers enum
16e225f70 Fix MergeContext::copied_operands_ strings moving
a4955b39a Run sandcastle tests in /dev/shm
ae0ad719d Fix flaky DBSSTTEST::DeleteObsoleteFilesPendingOutputs
b2a8016df Update db_bench_tool.cc (#1239)
c6654588b Disable two dynamic options tests under lite build
2a6d0cde7 Ignore stale logs while restarting DBs
ee8bf2e41 fixes 1228: rockdbjni loadLibraryFromJarToTemp fails when file is already present (#1232)
f85df120f Re-enable tsan crash white-box test with reduced killing odds
89e4c4882 Update README.md to include appveyor badge
b06ca5f86 ldb load, prefer ifsteam(/dev/stdin) to std::cin (#1207)
4ea0ab3cc Revert "Remove bashism from `make check` (#1225)"
12767b313 fixes 1220: rocksjni build fails on Windows due to variable-size array declaration (#1223)
a9d512a76 Update .gitignore for internal release
d5a51d4de Need to make sure log file synced before flushing memtable of one column family
89f319c2d Fix unit test which breaks lite build
b50632920 Add unit test not on /dev/shm as part of the pre-commit tests
b9a97181a Bump next release version
663afef88 Add EnvLibrados - RocksDB Env of RADOS (#1222)
32604e660 Fix flush not being commit while writing manifest
9ab38c45a Remove %z Format Specifier and Fix Windows Build of sim_cache.cc (#1224)
08ab1d83a Remove bashism from `make check` (#1225)
f9b14be49 Re-enable TSAN crash test but only with black box crash test
68f3eb746 Run release build for CLANG and GCC 4.8.1 in pre-commit tests too
e70020e4f Only cache level 0 indexes and filter when opening table reader
7bedd9440 Build break fix
68a8e6b8f Introduce FullMergeV2 (eliminate memcpy from merge operators)
e70ba4e40 MemTable::PostProcess() can skip updating num_deletes if the delta is 0
2a282e5f5 DBTablePropertiesTest.GetPropertiesOfTablesInRange: Fix Flaky
d9cfaa2b1 Persistent Read Cache (6) Persistent cache tier implentation - File layout
9430333f8 New Statistics to track Compression/Decompression (#1197)
515b11ffa fixes#1210: rocksdb/java/CMakeLists.txt lacks cmake_minimum_required (#1214)
876cb8bfb fixes#1212: rocksdbjni maven build does not escape slashes in groovy script (#1213)
21c55bdb6 DBTest.DynamicLevelCompressionPerLevel: Tune Threshold
4b9525358 Refactor cache.cc
c6a8665b3 Update LANGUAGE-BINDINGS.md
880ee363e ldb backup support
6797e6ffa Avoid updating memtable allocated bytes if write_buffer_size is not set
dda6c72ac Add DestroyColumnFamilyHandle(ColumnFamilyHandle**) to db.h
56222f57d Avoid FileMetaData copy
15b7a4ab8 Fixed output size and removed unneeded loop
6ea41f852 Fix deadlock when trying update options when write stalls
efd013d6d Miscellaneous performance improvements
e6f68faf9 Update Makefile to fix dependency
816ae098e fix test failure
e295da126 Fix Log() doc for default level
8e6b38d89 update DB::AddFile to ingest list of sst files
296545a2c Fix clang analyzer errors
61dbfbb6c Add release build to RocksDB per-diff/post-commit tests
907f24d0e Concurrent memtable inserter to update counters and flush state after all inserts
0f691c4b5 CLI option & Rename() allow overwrite
7c190070b delete unnessary pointer cast in beginInternalTransaction() function
e1b3ee8a7 Cleanup auto-roll logger flush-while-rolling test
cd4178a01 Add a new feature to enforce a sync point only active on a thread
b954847fc Fix release build for MyRocks by using debug-only code only in debug builds
a00bf1b3c Add More Logging to track total_log_size
01f77cb19 Update USER.md to include more services at Facebook.
eb53c05a3 Add comment for GetBackupInfo about returned BackupInfos order
32df9733d Add options.write_buffer_manager: control total memtable size across DB instances
5aaef91d4 group multiple batch of flush into one manifest file (one call to LogAndApply)
a45ee8318 Fix a bug that accesses invalid address in iterator cleanup function
38fae9e65 fix typos in HISTORY.md (#1192)
1a11c934d Disable some persistent cache tests on linux/clang
9b5adea97 Add More Logging to track total_log_size
95d96eeeb remove LockFile
ff45d1b54 if read only backup engine can't find meta dirs, return NotFound() instead of IOError()
cb2476a0c fix rate limiter to avoid starvation
6b7167651 Run env_basic_test on Env::Default
9eb0b5395 Move env_basic_test cleanup to TearDown
1fe3bf829 Re-enable linux on travis
43692793e Fixed Minor Bug on Windows Build and db_bench_tool.cc (#1189)
95c192475 writable file close before reset
197b832af Update USERS.md
bdb1d19a6 Fix UBSan build break caused by variable not initialized
b726bf596 FreeBSD does not have std::to_string (#1190)
faa7eb3b9 Improve regression_test.sh
c4cef07f1 Update DBTestUniversalCompaction.UniversalCompactionSingleSortedRun to use max_size_amplification_percent = 0
892e9d304 make transaction WriteOptions modifiable
4f2b0946d fix simple typos (#1183)
3b7ed677d ColumnFamilyOptions API [CF + RepairDB part 3/3]
56ac68629 Detect column family from properties [CF + RepairDB part 2/3]
3fc713ed9 delete 2nd level children for default env
343507afb Refactor to use VersionSet [CF + RepairDB part 1/3]
aa432be4b Workarounds for continuous build implementation
8cd9f04fe Test change to verify new commit detection
8a4ee7e90 Trivial change to test cont. build
af6248d8b Fix max_bytes_for_level_base comment
0d7b26123 add tests to env_basic_test.cc
6576fa05a Fix minor typos and PHP source file name used to trigger the builds
c4e19b77e Add a read option to enable background purge when cleaning up iterators
fa813f747 Update DB::AddFile() to ingest the file to the lowest possible level
d6b79e2fd Remove filter_deletes from crash_test
a52e4d7d0 Framework for enabling continuous RocksDB build and tests
f9bd66779 Makefile warning for invalid paths in make_config.mk
88a2776db Update SstFileWriter to use bottommost_compression if avaliable
e87d5df1a TiKV use-case (#1172)
7b79238b6 Deprectate filter_deletes
4939fc389 Bulk load mode shouldn't stop ingest
3a2bccc84 Fixed a crash bug that incorrectly parse deprecated options in options_helper
cf8adc971 Allow arcanist_util to work with both new and old arc versions
30a24f2d3 Add InternalStats and logging for AddFile()
d26a84807 Temporarily disable travis on linux
249e796df Fix Flaky DBCompactionTest.SkipStatsUpdateTest
162c9170d Make sandcastle access secure
8366e10ff Fix clang build
0babce57f Move away from enum char value -1
812dbfb48 Optimize BlockIter::Prev() by caching decoded entries
550bf895e Minor bug fix with log name
886af5910 Fix examples/Makefile jemalloc error
e3b1e3dfa Expose save points in Java WriteBatch and WBWI (#1092)
f5177c761 Remove wasteful instrumentation in FullMerge (stacked on D59577)
97fd2a638 Remove dead Jenkins code and support `arc diff --preview` in RocksDB
7c919decc Reuse TimedFullMerge instead of FullMerge + instrumentation
9a33a723b Remove the comments saying allow_concurrent_memtable_write and enable_write_thread_adaptive_yield are not stable
81f6b33d9 Fix tsan error
bc8af90e8 add option to not flush memtable on open()
8100ec2cd Fix libgcc broken lib path
7360db39e Add a check mode to verify compressed block can be decompressed back
2a79af1c5 Fix Java Break Related to memtable bloom bits to size ratio change
6faddd7c5 Merge db/slice.cc into util/slice.cc
5009b5326 BlockBasedTable::FullFilterKeyMayMatch() Should skip prefix bloom if full key bloom exists
2d05eaeb2 Fix name conflict in delete_shceduler_test and db_sst_test
bde7d1055 Fix clang_analyze path in fbcode_config.sh
ca3db5478 Fetch branches from github for format compatibility test
20699df84 memtable_prefix_bloom_bits -> memtable_prefix_bloom_bits_ratio and deprecate memtable_prefix_bloom_probes
e9c1face6 Minor fix to precommit-check.py
fcc47fa5f New features to precommit check script
56887f6cb Backup Options
a683d4aba URI-based Env selection for db_bench
53a4bd8a6 duplicate line
3e8686961 release 4.9 update version and history
b2973eaae Remove options builder
5b197bff4 Enabled Windows build for volatile tier implementation
281fbdddc Temporarily remove PersistentCacheOptions from persistent_cache_tier.h to fix unity build
2ae15b2d5 Format compatible test should cover forward compatibility up to 4.8.
00a058725 netflix use-case
5091dfc1e use branch names in format compatibility test
edc764e91 Use valgrind built with gcc-4.9-glibc-2.20
8ff59b2b4 Disable PersistentCacheTierTest.VolatileCacheInsertWithEviction test under TSAN temporarily
1ba452226 Fix for GCC 5.4 (#1157)
972c895c3 Previously WARN level logging became FATAL level logging in the Java API (#1089)
a73b26f60 Adding test for contiguous WAL detection
098da8348 Fix CLANG build break caused by the recent Persistent Cache change
54db29b8f Use gvfs links in dependencies.sh
d755c62f9 Persistent Read Cache (5) Volatile cache tier implementation
fda098461 Allow regression_test.sh to specify OPTIONS_FILE. Add header comments.
0fee89684 Fix Windows build
10d46b9c8 Update tp2 clang path to fix clang build
774a6aa29 Java API - Rename geHistogramData -> getHistogramData (#1107)
3070ed902 Persistent Read Cache (4) Interface definitions
e42dc9192 Update paths for fbcode dependencies
5647fa427 stack_trace,cc: The current Stacktrace code does not compile for FreeBSD (#1153)
0d65acec0 threadpool.cc: abort() lives in stdlib.h on FreeBSD (#1155)
19dd5a61c env_chroot.cc: FreeBSD likes stdlib.h for realpaht() and friends (#1154)
5aca977be env_basic_test library for testing new Envs [pluggable Env part 3]
1147e5b05 Adding support for sharing throttler between multiple backup and restores
6e6622abb Create env_basic_test [pluggable Env part 2]
e53287794 Add statistics field to show total size of index and filter blocks in block cache
a791a2cf2 Java API - Bugfix for native linking of Compaction Filter (#1099)
af0c9ac01 Env registry for URI-based Env selection [pluggable Env part 1]
02ec8154e allow updating block cache capacity from C (#1149)
630b732cb fix flaky sim_cache_test
62d548098 Add persistent cache to Windows build system
842958651 Fix race condition in SwitchMemtable
88acd932f Allows db_bench to take an options file
3a276b0cb Add a callback for when memtable is moved to immutable (#1137)
8cf0f86d3 Allow regression test to run db_bench at a remost host
27ad17071 Fix Windows build break
936973d14 Small tweaks to logging to track the number of immutable memtables
21c047ab4 add readahead size option (#1146)
71c7eed91 Assert boundary checks for SetPerfLevel()
5d85fdb2c add missing lock
c40c4cae1 LDBCommand::SelectCommand to use a struct as the parameter
590e2617e fix delete file bug when do checkpoint (#1138)
8dfa980cb Add statically-linked library for tools/benchmarks
f62fbd2c8 Handle overflow case of rate limiter's paramters
57461fba8 In-memory environment read beyond EOF
0e2000017 LDBCommand::InitFromCmdLineArgs() to move from template to function wrapper
472c06e90 Add low and upper bound values for rocksdb::PerfLevel enum
157e0633e MutexLock -> ThreadPoolMutexLock in util/threadpool.cc
23d4cf483 include/rocksdb/sst_file_writer.h should not depend on util/mutable_cf_options.h
345fd73fa Fix flaky DBTestDynamicLevel.DynamicLevelMaxBytesBase2
8fc75de32 Minor fix to disable DynamicLevelMaxBytesBase2
9dd50d990 Fix db_bench
5d660258e add simulator Cache as class SimCache/SimLRUCache(with test)
d379d110e Update CMakeLists.txt for added test
21f847eda Direct IO fix for Mac
99765ed85 Clean up the ComputeCompactionScore() API
def2f7bd0 Expose report_bg_io_stats option in the C API. (#1131)
f89caa127 Direct IO capability for RocksDB
8f1214531 C API: Expose DeleteFileInRange (#1132)
11f329bd4 db/db_impl: restrict WALRecoveryMode when using recycled log files
2b2a898e0 db/log_reader: combine kBadRecord{Len,Checksum} for readability
34df1c94d db/log_reader: treat bad record length or checksum as EOF
7947aba68 db/log_reader: move kBadRecord{Len,Checksum} handling into ReadRecord
847e471db db/log_test: add recycle log test
4e7e41ba7 Disable lite build/testing for persistent read cache
1d725ca51 Deprecate BlockBasedTableOptions.hash_index_allow_collision=false.
0e77246ba backupable_db.cc: lambada to explictly caputre "this" when escaping scope
2073cf377 Eliminate use of 'using namespace std'. Also remove a number of ADL references to std functions.
26adaad43 Split WinEnv into separate classes. (#1128)
bb98ca3c8 Implement GetUniqueId for Mac
1f2dca0ea Add MaxOperator to utilities/merge_operators/
f6e404c20 Added "number of merge operands" to statistics in ssts.
7383b64b3 Fix formatting of HISTORY.md (#1126)
0e665c399 Disable long running GroupCommitTest (#1125)
3c69f77c6 Move IO failure test to separate file
533cda90c Add GetStringFromCompressionType to include/rocksdb/convenience.h
c70a9335d Fix mutex unlock issue between scheduled compaction and ReleaseCompactionFiles()
05c5c39a7 Fix build
a6254f2bd Long outstanding prepare test
2ead11511 Fix TransactionTest.TwoPhaseMultiThreadTest under TSAN
1f0142ce1 Persistent Read Cache (Part 2) Data structure for building persistent read cache index
43afd72be [rocksdb] make more options dynamic
bac3be7c4 Fix build issue. (#1123)
f6aedb62c Fix Transaction memory leak
a08c8c851 Added PersistentCache abstraction
5c06e0814 [ldb] Templatize the Selector
aab91b8d8 Use generic threadpool for Windows environment (#1120)
a40033639 TransactionLogIterator sequence gap fix
fa3536d20 Store SST file compression algorithm as a TableProperty
40123b380 signed vs unsigned comparison fix
49815e384 [ldb] Export LDBCommandRunner
c1af07ce8 Disable backupable_db_test.cc on Windows
e61ba052b Isolate db env and backup Env in unit tests
560358dc9 Fix data race in GetObsoleteFiles()
5c1c90487 ldb option for compression dictionary size
c27061dae [rocksdb] 2PC double recovery bug fix
a657ee9a9 [rocksdb] Recovery path sequence miscount fix
8a66c85e9 [rocksdb] Two Phase Transaction
1b8a2e8fd [rocksdb] Memtable Log Referencing and Prepared Batch Recovery
0460e9dcc Modification of WriteBatch to support two phase commit
f548da33e Follow symlinks in chroot directory
d86f9b9c3 Fix lite build
4b3172343 Add bottommost_compression option
bfb6b1b8a Estimate pending compaction bytes more accurately
258459ed5 Properly destroy ChrootEnv in env_test
fca5aa6fc Initial script for the new regression test
e1951b6f2 Add --index_block_restart_interval option in db_bench
730f7e2e2 Fix win build
a9b3c47c8 Fix includes for clang on OS X
3f16a836a Introduce chroot Env
269f6b2e2 Revert "Modification of WriteBatch to support two phase commit"
04dec2a35 [ldb] Export ldb_cmd*.h
72c73cdc8 Java API - Add missing HEADER_LEVEL logging (#1104)
4d02bfa3a Add support for PauseBackgroundWork and ContinueBackgroundWork to the Java API (#1087)
8f65feafc Have sandcastle run lite_test for every diff
0d590d999 Make max_dict_bytes optional in options string
7ccb8d6ef BlockBasedTable::Get() not to use prefix bloom if read_options.total_order_seek = true
e3c6ba37d OptimizeForSmallDb(): revert some options whose defaults were just changed
967476eae Fix valgrind (DBIteratorTest.ReadAhead)
9790b94c9 Add optimize_filters_for_hits option to db_bench
a4ea345b0 Fixing lite build
24a24f013 Enable configurable readahead for iterators
ff4b3fb5b Fix Iterator::Prev memory pinning bug
cba752d58 sst_dump won't print size for unsupported compression type
6e801b0bd Eliminate memcpy in Iterator::Prev() by pinning blocks for keys spanning multiple blocks
1b166928c Release RocksDB 4.8.0
b8cf9130f Fix#1110, 32-bit build failure on Mac OSX (#1112)
21441c09b Fix calling GetCurrentMutableCFOptions in CompactionJob::ProcessKeyValueCompaction()
4ea6e051e Fix multiple issues with WinMmapFile fo sequential writing (#1108)
f3bb024fd Fix clang build
6e56a114b Modification of WriteBatch to support two phase commit
1d2e4ef74 ldb support new WAL records
a92049e3e Added EventListener::OnTableFileCreationStarted() callback
e8115cea4 Revert "Use async file handle for better parallelism (#1049)" (#1105)
6a14f7a97 Change several option defaults
c6c770a1a Use prefix_same_as_start to avoid iteration in FindNextUserEntryInternal. (#1102)
992a8f83b Not enable jemalloc status printing if USE_CLANG=1
029022b0f Fix crash_test
a06faa632 Skip PresetCompressionDict test for lite
e7899c661 Fix build issue. (#1103)
0f428c561 Fix compression dictionary clang osx error
6d4832a99 Merge pull request #1101 from flyd1005/wip-fix-typo
af70f9ac6 Fix typo in build_tools/fbcode_config.sh
54de13aba Fix compression dictionary clang errors
0850bc514 Fix build on machines without jemalloc
4032145ad Configurable compression in db_bench
843d2e313 Shared dictionary compression using reference block
ad573b902 Temporarily disable CompactFiles in db_stress in its default setting
1c80dfab2 Print memory allocation counters
eb7398085 Fix BackupableDBTest
ac0e54b4c CompactedDB should not be used if there is outstanding WAL files
d719b095d Introduce PinnedIteratorsManager (Reduce PinData() overhead / Refactor PinData)
1995e34d6 Retrieve file size from proper Env
7c14abf2c Improve BytewiseComparatorImpl::FindShortestSeparator
f3eb0b5b8 Make EventListenerTest.CompactionReasonLevel more deterministic
7b78d623f Shouldn't report default column family's compaction stats as DB compaction stats
995353e46 Fix null-pointer-dereference detected by Infer (https://github.com/facebook/infer)
24110ce90 Correct Statistics FLUSH_WRITE_BYTES
b71c4e613 Alpine Linux Build (#990)
90ffed1f9 Update USERS.md with link to LinkedIn blog post (#1088)
99a3bf8f6 Merge pull request #1068 from daaku/c-purge-old-backups
b54c34742 Use async file handle for better parallelism (#1049)
c146c9be1 rocksdb_create_mem_env to allow C libraries to create mem env (#1066)
6da70c581 expose more options in the c api (#1067)
6f01687aa C rocksdb_create_iterators to expose NewIterators (#1069)
644f978c1 Fix RocksDB Lite build in db_stress
5bd4022fe Add comparator, merge operator, property collectors to SST file properties (again)
7a6045a3c fix typo in HISTORY.md
73a847ef8 Add per-level compression ratio property
ee221d2de Introduce XPRESS compresssion on Windows. (#1081)
874c96ac1 Merge pull request #1083 from flabby/master
6cbffd50d Enable testing CompactFiles in db_stress
b95510ddf Fix DBTest.RateLimitedDelete flakiness
6356b4d51 Fix nullptr dereference in adaptive_table
9385fd72c Delete deprecated backup classes in Java
a2466c885 [db_stress] Make subcompaction random in crash_test
c3c389d54 Fix column label for L0 write sum
ec84bef24 New legocastle output parsing
725184b04 Fix db_block_cache_test in lite build
290883d94 Fix lite build
23089fd28 write_callback_test: clean test directory before running tests
792762c42 Split db_test.cc
40b840f29 Delete deprecated *BackupableDB interface for backups
6affd45d8 Make more tests run in parallel
47833e0ab Merge branch 'master' of github.com:facebook/rocksdb
e5c614e1d Fixing snapshot 0 assertion
1b1adebe8 Fixing snapshot 0 assertion
6d436a3f8 DBTest.HardLimit made more deterministic
994d9bc82 Make parallel valgrind watch-log more readable
9d35ae649 Make DBTestUniversalCompaction.IncreaseUniversalCompactionNumLevels more deterministic
cea8ed970 Fix backupable_db_test test cases that can't run by itself
4b6833aec Rename options.compaction_measure_io_stats to options.report_bg_io_stats and include flush too.
3894603fe Allow valgrind_check to run in parallel
c9d668c58 Fix unit tests issues on Windows (#1078)
083cadc7e Minor fix to Java sandcastle job definition
80b74a6c6 Include ldb tool in the windows build (#914)
7c14d11eb Minor fix to java build job
535af525d BlockBasedTable::PrefixMayMatch() to skip index checking if we can't find a filter block.
09be5cad5 Minor fix to sandcastle java job definition
1aeca9733 Release RocksDB 4.7
dfc3de8b7 Split Travis unittests Job
c2c8fe47f Add Java job for sandcastle
19ef3de57 Fix ManualCompactionPartial test flakiness
b345b3662 Add a minimum value for the refill bytes per period value
dff4c48ed BlockBasedTable::PrefixMayMatch: no need to find data block after full bloom checking
0353b853c Propagate sandcastle run error to UI
b885f33a5 Parallelize travis jobs
71303e04e Update db_bench_tool.cc (#1073)
63cf15bb9 Fix option settable tests
e208575b2 using java7 in runtime for hdfs env (#1072)
13e6c8e97 Relax an assertion in Compaction::ShouldStopBefore
ae21d71e9 Fixed a bug in RocksDB Statistics where flush is considered as compaction
8e0e22f76 Fix Windows build by replacing strings.h include
5675d5037 Revert travis commit
91f0f1f5e fix travis
a23c6052c Don't run DBOptionsAllFieldsSettable under valgrind
30d72ee43 PrefixTest.PrefixAndWholeKeyTest should run against a different directory from prefix_test
0e3cc2cf1 Add column family info to TableProperties::ToString()
2448f8037 Make sure that if use_mmap_reads is on use_os_buffer is also on
114a1b879 Fix build errors for windows
052299035 Improve sst_dump help message
0930e5e99 Update comments on include/rocksdb/perf_context.h
3b977bcdc instructing people to use java7 for hdfs (#1063)
1518b733e Change default number of cache shard bit to be 6 and max_file_opening_threads to be 16.
ada88b63f fix wrong assignment of level0_stop_writes_trigger in spatialdb (#1061)
2391ef721 Embed column family name in SST file
ab4c62332 Don't use version in the error message
d02eb8d00 Fix unused variable warning
9278097f8 Merge pull request #1056 from facebook/igorcanadi-patch-1
cc87075d6 No need to limit to 20 files in UpdateAccumulatedStats() if options.max_open_files=-1
8a1a603fd Eliminate std::deque initialization while iterating over merge operands
f38540b12 WriteBatchWithIndex micro optimization
200654067 Merge pull request #1053 from adamretter/benchmark-java-comparator
f2c43a4a2 Stderr info logger
b55e2165b Rocksdb backup can store optional application specific metadata
9b5198752 Adding pin_l0_filter_and_index_blocks_in_cache feature and related fixes.
2feafa3db Change some RocksDB default options
a558830f8 Fixed compile warnings in posix_logger.h and coding.h
51c9464df Merge pull request #980 from adamretter/java-arm
925b5d002 Merge pull request #1054 from DCEngines/magic12
63e8f1b55 Formatted lines to adhere to 80 char limit
994b3bd69 Add support for UBsan builds to RocksDB
21700a510 to/from hex refactor
24420947d Replace kHeader by WriteBatchInternal::kHeader in few more places kHeader was moved from write_batch.cc to header file because it is being used wherever the number "12" was being used to check for record size
3bdbe8961 Merge branch 'magic12' of https://github.com/dcengines/rocksdb into magic12
78711524b In all the places where log records are read, there was a check that record.size() should not be less than 12.
e7c64fb11 Imporve sst_file_manager comment
99ffb3d53 Fix perf_context::merge_operator_time_nanos calculation
07bb12d97 Update internal jemalloc and other versions
ad2fdaa82 Correct a typo in a comment
be9816b3d Fix data race issue when sub-compaction is used in CompactionJob
e3802531f Merge pull request #1050 from yuslepukhin/support_db_test2
e7cc49cbd Add support for db_test2 for dev and CI runs
3996770d0 Add comments to perf_context skip counters
4e85b7479 Make WritableFileWrapper not screw up preallocation
ec458dcde Merge pull request #1047 from PraveenSinghRao/wal_filter_ex
583157f71 Avoid overloaded virtual function
b9d4fa890 Options settable tests to use a different special charactor
60e34baef Merge pull request #1044 from PraveenSinghRao/wal_filter_ex
136b8e0ca Merge from master
2dcbb3b4f Addressed review comments
b1fafcaca Revert "Adding pin_l0_filter_and_index_blocks_in_cache feature."
5f8741a69 Revert "Fix failing Java unit test."
43bbb5619 tools/check_format_compatible.sh to use consistent version when testing backward and forward compatibility
d7ae42b0f Fix failing Java unit test.
fbbb8a614 Add test for Snapshot 0
e182f03c1 Add unit tests for RepairDB
7d371863e travis build fixes
fbea4dc66 Merge pull request #1042 from SherlockNoMad/HistFix
780d2b04c Update format compatible checking tool
4f1c74a46 merge from master
f8c218930 Publish log numbers for column family to wal_filter, and provide log number in the record callback
44756260a Reset block cache in failing unit test.
522de4f59 Adding pin_l0_filter_and_index_blocks_in_cache feature.
be2227126 Merge pull request #1041 from yuslepukhin/adjust_for_jemalloc
4ecc03c03 Fix in HistogramWindowingImpl
2ca0994cf Latest versions of Jemalloc library do not require je_init()/je_unint() calls. #ifdef in the source code and make this a default build option.
90aff0c44 Update --max_write_buffer_number for compaction benchmarks
72224104d Forge current file for checkpoint
33d568611 Merge pull request #1040 from bureau14/master
02e62ebbc Fixes warnings and ensure correct int behavior on 32-bit platforms.
9cad56861 Merge pull request #1039 from bureau14/master
3d29f9146 Improve documentation of the allow_os_buffer parameter.
3ff98bd20 Fix no compression test
b9cc42a72 Merge pull request #1038 from SherlockNoMad/HistFix
f76b260ef Fix FB internal CI build failure
774922c68 Merge pull request #1026 from SherlockNoMad/Hist
17b879b91 Merge pull request #1037 from SherlockNoMad/BuildFix
f11b0df12 Fix AppVeyor build error
6b03f93d4 Fix the build break on Ubuntu 15.10 when gcc 5.2.1 is used
697fab820 Updates to RocksDB subcompaction benchmarking script
58379bfb5 remove division from histogramwidowing impl
1a2cc27e0 ColumnFamilyOptions SanitizeOptions is buggy on 32-bit platforms.
e778c34e7 Merge pull request #1035 from bureau14/master
5bd3da1c5 Added quasardb to the USERS.md file
b2ae5950b Index Reader should not be reused after DB restart
0267655da Update change log for 4.6 release
08304c086 Expose RepairDB as ldb command
fd664a27b Fix Build Error
580fede34 Aggregate hot Iterator counters in LocalStatistics (DBIter::Next perf regression)
54f6b9e16 Histogram Concurrency Improvement and Time-Windowing Support
790252805 Add multithreaded transaction test
e8e6cf017 fix: handle_fatal_signal (sig=6) in std::vector<std::string, std::allocator<std::string> >::_M_range_check | c++/4.8.2/bits/stl_vector.h:794 #174
d9620239d Cleanup stale manifests outside of full purge
f71fc77b7 Cache to have an option to fail Cache::Insert() when full
ee8cc3520 Merge pull request #938 from alexander-fenster/master
765597fa7 Update compaction score right after CompactFiles forms a compaction
f0161c37b formatting fix
aa3f02d50 Improve comment in compaction.h and compaction_picker.h
2200295ee optimistic transactions support for reinitialization
badd6b784 Ignore db_test2
200080ed7 Improve snapshot handling for Transaction reinitialization
171c8e80b Update dependencies / Fix Clang
294bdf9ee Change Property name from "rocksdb.current_version_number" to "rocksdb.current-super-version-number"
bf1c4089d Use pure if-then check instead of assert in EraseColumnFamilyInfo
a7d4eb2f3 Fix a bug where flush does not happen when a manual compaction is running
68189f7e1 Update benchmarks used to measure subcompaction performance
dfe96c72c Fix WriteLevel0TableForRecovery file delete protection
451678c8c Merge pull request #1025 from SherlockNoMad/BuildFix
58ecd9132 Fix Windows build
501927ffc [backupable db] Remove file size embedded in name workaround
ef204df7e Compaction always needs to be removed from level0_compactions_in_progress_ for universal compaction
e79ad9e18 Add Iterator Property rocksdb.iterator.version_number
19ea40f8b Subcompaction boundary keys should not terminate after an empty level
deb08b822 Add parsing of missing DB options
f8e90e875 Get file attributes in bulk for VerifyBackup and CreateNewBackup
12fd9b186 Change BlockBasedTableOptions.format_version default to 2
4572a2d8c Update current version to 4.6
74b660702 Rename iterator property "rocksdb.iterator.is.key.pinned" => "rocksdb.iterator.is-key-pinned"
6743135ea Fix DB::AddFile() issue when PurgeObsoleteFiles() is called
432f3adf2 Add DB Property "rocksdb.current_version_number"
188bb2e7a Fix formatting identified by `arc lint`
0f2d2fcff Refactored tests to use try-with-resources
f8e02c782 Deprecate org.rocksdb.AbstractNativeReference#dispose() and implement java.lang.AutoCloseable
0f2fdfe23 Fix the javadoc and the formatting of the base-classes for objects with native references
b5b1db167 Recompute compaction score after scheduling manual compaction
5ea9aa3c1 TransactionDB:ReinitializeTransaction
1f5954147 Introduce Iterator::GetProperty() and replace Iterator::IsKeyPinned()
67789419f Merge pull request #1020 from gongsu832/master
69c471bd9 Handle concurrent manifest update and backup creation
3373c81fa Modify build_tools/build_detect_platform to detect and set -march=z10 on Linux s390x.
990509045 Merge branch 'master' of https://github.com/gongsu832/rocksdb
3492889ab Merge pull request #1019 from javacruft/wip-omit-leaf-frame-pointer-archs
7ca731b12 build: Improve -momit-leaf-frame-pointer usage
21f17aaa6 Modified Makefile and build_tools/build_detect_platform to compile on Linux s390x.
8800975fb Make DBTestUniversalCompaction.IncreaseUniversalCompactionNumLevels more robust
cd3fe675a Remove stale TODO
69c98f043 Reorder instance variables in backup test for proper destruction order
82f15fb15 Add test to make sure DropColumnFamily doesn't impact existing iterators
38201b359 Fix assert failure when DBImpl::SyncWAL() conflicts with log rolling
2568985ab IOStatsContext::ToString() add option to exclude zero counters
b04691665 Redo SyncPoints for flush while rolling test
291ae4c20 Revert "Revert "Fixed the bug when both whole_key_filtering and prefix_extractor are set.""
eef63ef80 Fixed CompactFiles() spuriously failing or corrupting DB
79ca039eb Relax the check condition of prefix_extractor in CheckOptionsCompatibility
4b1b4b8ae Merge pull request #1004 from yuslepukhin/child_attr
9ea2968d2 Implement ConsistentChildrenAttribute by using default implementation for now as it works.
c7f1a8a46 Fix LITE build thread_local_test
0914f0ca5 Merge pull request #1003 from yuslepukhin/fix_mutexlock_pthread_build
d37d348da This addresses build issues on Windows https://github.com/facebook/rocksdb/issues/1002
d825fc70d Use condition variable in log roller test
6b2a047df Fix SstFileManager uninitialized data member
a3db93c26 Remove the SyncPoint usage in the destructor of PosixEnv
df9ba6df6 Introduce SstFileManager::SetMaxAllowedSpaceUsage() to cap disk space usage
3943d1678 Fix race conditions in auto-rolling logger
d733dd572 [build] Fix env_win.cc compiler errors
cf38e56f2 Fix broken appveyor build caused by D53991
351252b49 Merge pull request #998 from fengjian0106/master
133605249 fix ios build error
d08d50295 Fix transaction locking
730a422c3 Improve the documentation of LoadLatestOptions
a7b6f0748 Improve write_with_callback_test to sync WAL
5bcf952a8 Fix WriteImpl empty batch hanging issue
871cc5f98 fix build without gflags
c90d63a23 can_unlock set but not used
44371501f Fixed a segfault when compaction fails
2f084d39b Merge pull request #992 from jofusa/jdennison/options-typo-fix
7bd284c37 Separeate main from bench functionality to allow cusomizations
bd5f842bb fixes typo in options logging
1c868d684 Fix includes for env_test
545a19395 Add J to commit_prereq so comilation/execution happens in parallel
5bb7371ca [build] Evaluate test names only when db_test exists
6a2b4fcb8 Add flag to forcibly disable fallocate
92a9ccf1a Add a new compaction priority that picks file whose overlapping ratio is smallest
3dc3d1c14 Merge pull request #984 from petermattis/pmattis/comparator-iterate-upper-bound
239aaf2fc Use user_comparator when comparing against iterate_upper_bound.
908100399 Fixed a dependency issue of ThreadLocalPtr
337671b68 Add universal compaction benchmarks to run_flash_bench.sh
3a67bffaa Fix an ASAN error in transaction_test.cc
c5af85eca Fix a memory leak of Slice objects from org.rocksdb.WBWIRocksIterator#entry1
e84137c8a Remove unnessecary java.util.List expense in JNI
76e8beeeb Pass by pointer from/to Java from JNI not by object
0e7e6f6e4 Improve Javadoc
18eb56305 Improve the speed and synchronization around the construction of Java/JNI objects
2a04268be Temporarily disable unstable tests in memory_test.cc
08a78b6b4 Merge pull request #979 from facebook/update_licenses
21e95811d Updated all copyright headers to the new format.
59b3ee658 Env function for bulk metadata retrieval
4a8cbf4e3 Allows Get and MultiGet to read directly from SST files.
fe93bf9b5 Transaction::UndoGetForUpdate
2608219cc crash_test: cover concurrent memtable insert in default crash test
a76e9093f Fix LITE db_test build broken by previous commit
b1887c5dd Explictly fail when memtable doesn't support concurrent insert
8ed343877 Add option to run fillseq with WAL enabled in addition to WAL disabled
73a9b0f4b Update version to 4.5
6f71d3b68 Improve perf of Pessimistic Transaction expirations (and optimistic transactions)
8e6172bc5 Add BlockBasedTableOptions::index_block_restart_interval
34a40bf91 Add --allow_concurrent_memtable_write in stress test and run it in crash_test
73bf330c7 Merge pull request #973 from yuslepukhin/moveout_testcode
f7c0f4e3e perf_context.cc and iostats_context.cc use different output macro (fix unity build)
9656eab00 This partially addresses issue https://github.com/facebook/rocksdb/issues/935 testutil.cc and testharness.cc could not be moved out at this time as they are used by 4 benchmarks in release builds.
14a322033 Remove references to files deleted in commit abb405227848581d3e6d2ba40d94dbc0a5513902
8445e5380 Add a mechanism to run all tests in sandcastle
461cec4e8 Merge pull request #972 from adamretter/wb-threads
9ab269ab3 Threaded tests for WriteBatch
bf767c641 Minor fix to makefile
2c1db5ea5 always invalidate sequential-insertion cache for concurrent skiplist adds
c12ff20ab Merge pull request #965 from koldat/jni_for_windows
a09ce4fcd Skip some of the non-critical tests in ./tools/run_flash_bench.sh
284aa613a Eliminate duplicated property constants
94be872ea Merge branch 'master' of github.com:facebook/rocksdb
0c2bd5cb4 Removing data race from expirable transactions
5fcd1ba30 disable kConcurrentSkipList multithreaded test
466c2c1bf Generate tags for *.c files
70c068c97 Merge pull request #960 from koldat/masterFixes2
a62c519bb RollLogFile tries to find non conflicting file until there is no conflict.
57a95a700 Making use of GetSystemTimePreciseAsFileTime dynamic - code review fixes
502d41f15 Making use of GetSystemTimePreciseAsFileTime dynamic to not break compatibility with Windows 7. The issue with rotated logs was fixed other way.
52153930d Adding support for Windows JNI build - fix Java unit test for release build of JNI DLL
e2972803a Adding support for Windows JNI build
9c2cf9479 Fix for --allow_concurrent_memtable_write with batching
ac3fa9a6f Travis CI to disable ROCKSDB_LITE tests
7b943da1b Merge pull request #967 from SherlockNoMad/ValueSize
b5750790e Merge pull request #968 from yuslepukhin/one_shot_buffer
1ad818295 Fix WriteBatchTest.ManyUpdates, WriteBatchTest.LargeKeyValue under clang
ad7ecca72 Add unit tests to verify large key/value
fdd70d149 Skip filters for last L0 file if hit-optimized
aa5e3b7c0 PerfContext::ToString() add option to exclude zero counters
36300fbbe Enable per-request buffer allocation in RandomAccessFile This change impacts only non-buffered I/O on Windows. Currently, there is a buffer per RandomAccessFile instance that is protected by a lock. The reason we maintain the buffer is non-buffered I/O requires an aligned buffer to work. XPerf traces demonstrate that we accumulate a considerable wait time while waiting for that lock. This change enables to set random access buffer size to zero which would indicate a per request allocation. We are expecting that allocation expense would be much less than I/O costs plus wait time due to the fact that the memory heap would tend to re-use page aligned allocations especially with the use of Jemalloc. This change does not affect buffer use as a read_ahead_buffer for compaction purposes.
1d854fa3d Fixed the asan error on column_family_test
37159a644 Add histogram for value size per operation
3b2a1ddd2 Add options.base_background_compactions as a number of compaction threads for low compaction debt
6ee38bb15 Slowdown of writing to the last memtable should not override stopping
d6c838f1e Add SstFileManager (component tracking all SST file in DBs and control the deletion rate)
45768ade4 transaction allocation perf improvements
77926f93e Merge pull request #964 from benoitc/fix/pi2
03a5661a1 fix build for raspberry 2
4b50f1354 Should not skip bloom filter for L0 during the query.
eadd221d3 Merge pull request #959 from koldat/master
d209076fa Merge pull request #961 from wingify/master
26c618004 Add Wingify to USERS.md
4265f81e8 Remove util/auto_roll_logger.cc (it was moved to different directory)
d7f22b6d2 Fixing generated GenerateBuildVersion.vcxproj when one builds on different locale than english. The problem is that date and time CLI utilities generates different format so that REGEX in CMake does not work.
d20915d52 Disable stats about mutex duration by default
0c433cd1e Fix issue in Iterator::Seek when using Block based filter block with prefix_extractor
035857a31 Fix RocksDB lite build
77ef87ccb Update fbcode_config4.8.1.sh to use update_dependencies.sh
955ecf8b4 Fix an ASAN error in compact_files_test
b0afcdeea Fix bug in block based tables with full filter block and prefix_extractor
167bd8856 [directory includes cleanup] Finish removing util->db dependencies
acd7d5869 [directory includes cleanup] Remove util->db dependency for ThreadStatusUtil
46f9cd46a [directory includes cleanup] Move cross-function test points
22ecb752d Add valgrind to pre-commit sandcastle testing
b7ecf3d21 Fix intermittent hang in ColumnFamilyTest.FlushAndDropRaceCondition
38e1d7fea ldb to support --column_family option
da33dfe18 Parameterize DBTest.Randomized
fb9811ee9 Add a perf context level that doesn't measure time for mutex operations
f7ef1a613 Include rest of dependencies in dependencies.sh
3e9209a07 Updated GetProperty documentation
40911e0b3 Run unit tests in parallel to find failing tests
2fbc59a34 Disallow SstFileWriter from creating empty sst files
f53c95f81 Cosmetic fixes and comments for the reader
f1ed17010 Add tests to make sure new DB or ColumnFamily options are settable through string
f57596b0c Improvements to pre-commit
538eec066 Update fbcode_config.sh to use latest versions automatically
8019aa9b5 improve test for manifest write failure
bcd4ccbc3 Revert D7809
b0a15e7fb Mechanism to run CI jobs on local branch via commit_prereq
bb2888738 Cleanup property-related variable names
29289333d Add named constants for remaining properties
2c2b72218 Disable OptionsParserTest.BlockBasedTableOptionsAllFieldsSettable under CLANG
a300d9928 Added sandcastle pre-commit
202be23e4 Add test that verifies all options in BlockBasedTableOptions is settable through GetBlockBasedTableOptionsFromString()
eceb5cb1b Split db_test.cc (part 1: properties)
94918ae84 db_bench: explicitly clear buffer in compress benchmark
fdbff4239 Crash test to make kill decision for every kill point
39c3e94ff Merge pull request #954 from davidbernard/solaris_build
df7c2f3b5 As per google coding standard replace "using" in option_builder.cc and geodb_impl.cc
12809b44b Revert "Change notification email for travis"
34704d5c7 [easy] Fixed a crash in LogAndApply() when CF creation failed
791dbafa9 Merge pull request #953 from sselva/master
594a5ef02 Merge pull request #955 from bcbrock/ppc64-build
f423f05dc Simple changes to support builds for ppc64[le] consistent with X86
3f12e16f2 Make alloca.h optional
eaa563756 Change notification email for travis
d78c6b28c Changes for build on solaris
2e9fae3f2 Add Rakuten Marketing to USERS.md
83e1de92a move internal build to use zstd 0.4.5
aec10f734 Guard falloc.h inclusion to avoid build breaks
f7ebc2f34 Update HISTORY.mc for 4.4.0
addd9545f Merge pull request #947 from yuslepukhin/align_and_yield
ac50fd3a7 Align statistics Use Yield macro to make it a little more portable between platforms.
b54d4dd43 tools/sst_dump_tool_imp.h not to depend on "util/testutil.h"
48a8667c3 Merge pull request #929 from warrenfalk/fix32
d9bca1e14 Reduce iterator deletion overhead
45d794068 Merge pull request #940 from yuslepukhin/fix_windows_build_signed_unsigned
20d7902df Fix compile error. Use constructor style initialization instead of a cast for simplicity.
e16438bb8 fixing build warning
b73fbbaf6 added --no_value option to ldb scan to dump key only
1477dcb37 Merge pull request #937 from petehunt/master
c7cb1076a Add Smyte to USERS.md
df7e3b622 Include <array> in table/plain_table_key_coding.h
235b162be Not scheduling more L1->L2 compaction if L0->L1 is pending with higher priority
9a8e3f73e plain table reader: non-mmap mode to keep two recent buffers
7ece10ecb DeleteFilesInRange: Mark files to be deleted as being compacted before applying change
94d9df248 fix an unused function compiler warning in crc32c in 32-bit mode
2f01e10fa use static_cast in crc32c instead of c-style cast
601f1306a fix shorten-64-to-32 warning in crc32c
f3fb39814 Fix BlockBasedTableTest.NoopTransformSeek failure
55b37efa1 fix a compile error on 32-bit
8c71eb5af Optimize DBIter::Prev() by reducing stack overhead
73c31377b Revert "Fixed the bug when both whole_key_filtering and prefix_extractor are set."
57605d7ef Fixed the bug when both whole_key_filtering and prefix_extractor are set.
6935eb24e Add ColumnFamilyHandle::GetDescriptor()
9760c842c fix valgrind failure in backupable_db_test
b1a3b4c0d Make ldb automagically determine the file type and use the correct dumping function
ba8344736 Merge pull request #923 from petermattis/pmattis/prefix-may-match
da032495d Optimize GetLatestSequenceForKey
260c29762 Fix index seeking in BlockTableReader::PrefixMayMatch.
e541dcc8f Fix issue #921
51adc5457 fix sporadic failure in fault_injection_test
a2422f053 fix potential test SleepingTask race condition
1627c4b1b Merge pull request #918 from mkurdej/fix/assertion-on-no-disk-space
92d0850f1 Fix failing assertion in logger on Windows when the disk is full.
7699439b7 Prevent the user from setting block_restart_interval to less than 1
4041903ec Enhance db_bench write rate limit
399343205 Add Airbnb and Pinterest to USERS.md
d74c9f0a5 DeleteFilesInRange: Clean job context if no files deleted
1dec5b8f5 Merge pull request #916 from warrenfalk/capi_huge_page_option
12fa27b4f Merge pull request #915 from warrenfalk/capi_full_bloom
0fde291ab expose memtable_prefix_bloom_huge_page_tlb_size option to C API
7e81dba5c Support creation of "full" format bloom filter from C API
bae5b0a1d Fix clang build
ac16663bd use -Werror=missing-field-initializers, to closer match MyRocks build
ab5a9a66d Merge pull request #911 from shuzhang1989/fix_envhdfs_virtual_func
eb5a13904 update posix env schedule call
a41f68ac2 fix inconsistency between env_hdfs and env
7238be090 Fix clang build in db_compaction_test
c9e2490bc Fix DynamicBloomTest.concurrent_with_perf to pass TSAN
63ddb783d Delete files in given key range
d8677a8d2 Upgrade internal CLANG version for FB-internal gcc 4.8.1
edf1cd497 Not generating "__attribute__((__unused__))" for padding fields if it is not CLANG
9eb4f9596 Merge pull request #907 from siying/master
22c0ed8a5 Disable Visual Studio Warning C4351
fcafac053 Fix memory leak in ColumnFamilyTest.WriteStall*
b99d4276f Fix java test buid broken by 7d87f02799bd0a8fd36df24fab5baa4968615c86
11672df19 Fix CLANG errors introduced by 7d87f02799bd0a8fd36df24fab5baa4968615c86
7fafd52dc Merge pull request #900 from shuzhang1989/hdfs_env_fix
2b7c810db more foramt
b79ccbd57 indent
7d87f0279 support for concurrent adds to memtable
5b2587b5c DBTest.HardLimit use special memtable
b4aa82366 format
4dfdd1d92 format
298ba27ae Merge pull request #846 from yuslepukhin/enble_c4244_lossofdata
7810aa802 Merge pull request #899 from zhipeng-jia/fix_clang_warning
4c5560d70 Merge pull request #895 from zhipeng-jia/develop
d43da8ae0 DBTest.DelayedWriteRate: fix assert of sign and unsign comparison
3280ae9a2 Fix warning in release
ec2664fef Fix clang compile error under Linux
4fd23fb13 add a factory method for creating hdfs env
9c176ef90 Update liblz4 to r131
15b890226 Change default options.delayed_write_rate
73b175a77 Fix clang warnings regarding unnecessary std::move
b9f77ba12 When slowdown is triggered, reduce the write rate
445d5b8c5 Fix clang build
e089db40f Skip bottom-level filter block caching when hit-optimized
06c05495e Merge pull request #898 from zhipeng-jia/fix_move_warning
aa515823b Fix clang warning
2ba03196d Merge pull request #897 from yuslepukhin/enable_status_move
dbb8260f7 Make Status moveable Status is a class which is frequently returned by value from functions. Making it movable avoids 99% of the copies automatically on return by value.
2bf9b968c Fix lite_build
d005c66fa Report compaction reason in CompactionListener
728f944f0 Fix computation of size of last sub-compaction
8ac7fb837 Merge pull request #863 from zhangyybuaa/fix_hdfs_error
e53e8219a Merge pull request #894 from zhipeng-jia/develop
e0abec158 Sorting std::vector instead of using std::set
33e09c0e1 add call to install superversion and schedule work in enableautocompactions
22c6b50ee Merge pull request #893 from zhipeng-jia/develop
24c7dae13 Fix clang warning regarding implicit conversion
eff309867 Do not use timed_mutex in TransactionDB
97ea8afaa compaction assertion triggering test fix for sequence zeroing assertion trip
521da3abb Fix BlockBasedTableTest.BlockCacheLeak valgrind failure
a48382399 Fix use-after free in db_bench
bf8ffc1d6 Merge pull request #890 from zhipeng-jia/develop
131f7ddf6 fix typo: sr to picking_sr
c37729a6a db_bench: --soft_pending_compaction_bytes_limit should set options.soft_pending_compaction_bytes_limit
7b12ae97d Add signalall after removing item from manual_compaction deque
d72b31774 Slowdown when writing to the last write buffer
6b2a3ac92 Add documentation for unschedFunction
167fb919a ZSTD to use CompressionOptions.level
32ff05e97 Bump version to 4.4
aececc209 Introduce ReadOptions::pin_data (support zero copy for keys)
e6e505a4d Fix examples
aa29cc128 Improve examples/README.md
97265f5f1 Fix minor bugs in delete operator, snprintf, and size_t usage
b68dc0f83 Merge pull request #885 from yuslepukhin/fix_size_t_formatting
b6d19adcf Use port size_t formatting
963660eb5 Merge pull request #883 from zhipeng-jia/master
99ae549d3 Fix typo
636cd3c71 Clean up listener_test (reuse db_test_util)
030215bf0 Running manual compactions in parallel with other automatic or manual compactions in restricted cases
d26a4ea62 Merge pull request #882 from SherlockNoMad/BuildFix
768a61486 Fix appVeyor Build problem
aca403d2b Fix another rebase problems.
a6fbdd64e Fix rebase issues and new code warnings.
3fa68af31 Enable MS compiler warning c4244. Mostly due to the fact that there are differences in sizes of int,long on 64 bit systems vs GNU.
236fe21c9 Enable MS compiler warning c4244. Mostly due to the fact that there are differences in sizes of int,long on 64 bit systems vs GNU.
84f98792d Transaction::SetWriteOptions()
3bfd3d39a Use SST files for Transaction conflict detection
362d819a1 Improving parser
00d6edf6a Ensure the destruction order of PosixEnv and ThreadLocalPtr
64fa43843 Merge pull request #862 from ceph/wip-env
2074ddd62 env: add EnvMirror
a3ba5915c Correct a comment in include/rocksdb/cache.h
f0a8e5a2d Fixed the valgrind error in ColumnFamilyTest::CreateAndDropRace
9e4462906 Change SingleDelete to support conflict checking
c5af8bffb Merge pull request #879 from charsyam/feature/typos
c30b49954 fix typos in comments
56e77f096 Deprecate options.soft_rate_limit and add options.soft_pending_compaction_bytes_limit
d6e1035a1 A new compaction picking priority that optimizes for write amplification for random updates.
de6958b2e Merge pull request #877 from yuslepukhin/fix_unnecessary_type_truncation
0991cee6c Merge pull request #815 from SherlockNoMad/CounterFix
49957f9a9 Prefer integer arithmetics The code had conversion to double then casting to size_t and then casting uint32_t which caused compiler warning (VS15).
0836d265c Merge pull request #876 from warrenfalk/wf_win_master
c6fedf2bf Add compaction_iterator and delete_scheduler tests to Windows build
ac8e56f05 db_bench: in uncompress benchmark, get Snappy size from compressed stream
9c227923c Merge pull request #788 from OpenChannelSSD/to_fb_master2
fa3dbf203 Merge pull request #853 from Vaisman/enable_C4267_warning
ad6aaf4fa Merge pull request #848 from SherlockNoMad/db_bench
56bbecc31 Merge pull request #867 from SherlockNoMad/CacheFix
188170fb4 Updating HISTORY.md
758dbec7f Fix fb-only build for gcc 4.8.1
774b80e99 Resubmit the fix for a race condition in persisting options
afc84731f Include ldb_tools and sst_dump_tools libraries in shared library
e5c5f2381 Support marking snapshots for write-conflict checking - Take 2
ea1192355 Upgrade to ZSTD 0.4.2
b60cb88c7 Update examples/rocksdb_option_file_example.ini
3d8bb2c89 Fix valgrind failure in IncreaseUniversalCompactionNumLevels
7af91d425 Merge pull request #873 from yuslepukhin/make_vs15_build
1d63c3d61 Revert "Support marking snapshots for write-conflict checking"
78de0c922 Fix up VS 15 build. Fix warnings Take advantage of native snprintf on VS 15
ec704aafd Support marking snapshots for write-conflict checking
770dea932 Fix occasional failure of DBTest.DynamicCompactionOptions
ebc2d490d Split histogram per OperationType in db_bench
f307036bd Revert "Fix a race condition in persisting options"
2fa3ed518 Fix a race condition in persisting options
f276c3a82 Fix valgrind failures in 3 tests in db_compaction_test due to new skiplist changes
291088ae4 Fix undeterministic failure of ColumnFamilyTest.DifferentWriteBufferSizes
3c2b995fb Merge branch 'master' of https://github.com/facebook/rocksdb into CacheFix
355fa9436 EstimatedNumKeys Counter Inaccurate
b2863017b Move posix threads into a library
3a98a7ae7 Replace malloc with new for LRU Cache Handle
a9ca9107b Fix db_universal_compaction_test
d3bb572da Build break fix.
b28b7c6dd Added callback notification when a snapshot is created
e8180f990 added public api to schedule flush/compaction, code to prevent race with db::open
19b1201b2 Merge pull request #865 from yuslepukhin/fix_db_table_properties_test
e0de7ef87 Avoid empty ranges vector with subsequent zero element access
a330f0b3b Fix incorrect merge in db/db_compaction_test.cc
bd7a49d44 Make DBCompactionTestWithParam::CompactionTrigger more deterministic
be006d288 fix LinkFile() undefined reference error
4687ced5d fix ToString() not declared error
bcd7bd122 Relax verification condition of DBTest.SuggestCompactRangeTest
f9103d9a3 DBTest.DynamicCompactionOptions: More deterministic and readable
0ad68518b Fix DBCompactionTestWithParam.CompactionTrigger in non-jemalloc build.
459c7fba3 Revert previous behavior of internal_key_skipped_count
481f9edb1 Fix CLANG build
d7421c22f Fixed some typos in the comments of rocksdb options file example
ef8ed3681 Fix DBTest.SuggestCompactRangeTest for disable jemalloc case
db320b1b8 DB to only flush the column family with the largest memtable while option.db_write_buffer_size is hit
4a009f917 Merge pull request #860 from SherlockNoMad/BuildFix
b4efaebff Fix ms version Appveyor build error
d27ea4c9e Initialize options.row_cache
72930485b Fix clang build
6bbfa1874 BackupDB to have a mode to use file size in file name
f3ea00bc8 Merge pull request #856 from ceph/wip-env
4cedd6b03 EnvWrapper: add ReuseWritableFile
33e0c9382 Reduce extra key comparision in DBIter::Next()
9a9d4759b InlineSkipList part 3/3 - new skiplist type that colocates key and node
520172954 InlineSkipList - part 2/3
78812ec6b InlineSkipList - part 1/3
ffb466da4 Merge pull request #855 from yuslepukhin/enable_3rdparty_override
10d257d64 Enable override to 3rd party linkage
41b32c605 Enable C4267 warning
c5b467306 Fix race condition that causes valgrind failures
efb01a055 Merge pull request #850 from yuslepukhin/enable_2015_build
81be49c75 Have a way for compaction filter to ignore snapshots
047bd22aa Build on Visual Studio 2015 Update 1
88e052772 Reduce moving memory in LDB::ScanCommand
890f44f46 Merge pull request #844 from yuslepukhin/enable_C4804_unsafe_bool
9d0b8f19d plain table reader: avoid re-read the same position for index and data in non-mmap mode
89bacb7e7 Enable MS Warning C4804 : unsafe use of type 'bool' in operation
d5239f870 build_tools/fbcode_config4.8.1.sh: upgrade versions of some dependencies
c4ebb66d6 Not to build forward_iterator_bench now
c342549d0 Merge pull request #841 from yuslepukhin/fix_test_rerun_logic
7cb1293b6 Fix log names when scheduling runs and reruns
51fce92e1 "ldb compact" should force bottommost level compaction
f83164120 Merge pull request #837 from yuslepukhin/rerun_concurrency_value
4159ab816 Merge pull request #839 from SherlockNoMad/memtableOption
6170fec25 Fix build broken by previous commit of "option helper refactor"
3a6643c2f Merge pull request #805 from SherlockNoMad/OptionHelperFix
189b3e03d Fix uninitilizeded SpecialEnv::time_elapse_only_sleep_
d5540e18e DBTest.MergeTestTime to only use fake time to be determinstic
bd7be035e Support Memtable Factory Parse in option_helper.cc
94e39e236 Exclude DBTest.FileCreationRandomFailure as a long running test Increase concurrency to 18 Fix exclusion but in the ps script
4189c0f9a Fix Java Makefile
605a24d94 Block forward_iterator_bench under MAC and Windows
2a0510c9f Failed tests must be rerun with concurrency 1
9b8c9be0b Fix forward_iterator allocation of vector.
5cbb7e43e DBTest.MergeTestTime: relax counter upper bound verification
52e04b3d0 Merge pull request #833 from yuslepukhin/fix_win_build_after_lint
314f62194 Remove headers from the cc since they are in the module's header.
472c74006 Add necessary headers after cpplint rearranged includes
9bc9c93bd Move to version 4.3
3381e2c3e Handle multiple calls to DBImpl::PauseBackgroundWork() and DBImpl::ContinueBackgroundWork()
65a042921 Merge pull request #831 from yuslepukhin/remove_forward_iter_bench_win
ca5566d20 Fix clang build
4175472ad Merge pull request #832 from yuslepukhin/fix_forward_iter_outofbounds
cb9459f85 Fix empty vector write in ForwardIterator
a163cc2d5 Lint everything
8f01f2541 Remove forward_iter_bench from Win build.
dac5b248b UniversalCompactionPicker::PickCompaction(): avoid to form compactions if there is no file
d06b63e99 Fix Rocksdb lite build failure in forward_iterator_bench
7824444bf Reuse file iterators in tailing iterator when memtable is flushed
2ae4d7d70 Make sure that CompactFiles does not run two parallel Level 0 compactions
d781da816 Add CheckOptionsCompatibility() API to options_util
2391b459b Merge pull request #824 from yuslepukhin/try_ci_tests_on_daily
2ab3e2df2 Fix a build break so tests can run
247c49a40 Merge branch 'master' into try_ci_tests_on_daily
935d1495c Run tests imporvements Add sequential rerun for any failed tests. Add env_test case. Limit concurrency Allow to specify individual tests Take $Limit into account when displaying number of tests
5ac16300b Fixed valgrind error in options_util_test
6ce42dd07 Don't merge WriteBatch-es if WAL is disabled
56245ddcf Fixed DBCompactionTest.SkipStatsUpdateTest
e11f676e3 Add OptionsUtil::LoadOptionsFromFile() API
e78389b55 Fixed build failure of RocksDBLite test on options_file_test.cc
e114f0abb Enable RocksDB to persist Options file.
7ed2c3e45 Merge pull request #823 from yuslepukhin/fix_off_t_type
7f59e33b1 Make CI build debug/optimized
ae2dfe404 Try running db_test during integration build
720af2269 Merge branch 'fix_off_t_type' of https://github.com/yuslepukhin/rocksdb into fix_off_t_type
5270b33bd Make use of portable `uint64_t` type to make possible file access in 64-bit.
631863c63 track WriteBatch contents
505accda3 remove constexpr from util/random.h for MSVC compat
5421c9728 Make use of portable `uint64_t` type to make possible file access in 64-bit.
b81b43098 Switch to thread-local random for skiplist
75a8bad2a Merge pull request #821 from yuslepukhin/continue_windows_warnings
986230b8c Revert "Fix TSAN build for fbcode"
f3ca28ab0 Correct the comment of GetApproximateMemoryUsageByType
838676c17 Revert "Adding new table properties"
7c86d5049 Enable C4305 'identifier' : truncation from 'type1' to 'type2'
85a2ce9c1 Enable C4702 unreachable code
62aa1b1b7 Enable C4200 warning nonstandard extension used : zero-sized array in struct/union
5b9ce1a32 Merge pull request #820 from yuslepukhin/enable_compiler_warnings
20f57b171 Enable Windows warnings C4307 C4309 C4512 C4701 Enable C4307 'operator' : integral constant overflow Longs and ints on Windows are 32-bit hence the overflow Enable C4309 'conversion' : truncation of constant value Enable C4512 'class' : assignment operator could not be generated Enable C4701 Potentially uninitialized local variable 'name' used
8be568a9c Adding new table properties
2b42000f4 incorrect batch group size computation for write throttling
c745f1d2c Fix TSAN build for fbcode
fe789c5f2 Document SingleDelete
e89e5b253 Merge pull request #818 from yuslepukhin/improve_test_concurrency
ae7940b62 Fix regression failure in PrefixTest.PrefixValid
3277d172b Improve concurrency when running tests PowerShell seems to have a hard time when a flood of async tasks is scheduled at the same time. I speculated that WaitForMultipleObjects() in Windows can only take up to 64 process handles and if you want to handle more than you should write some additional code which can be sub-optimal. I.e to implement Wait-Job -Any. I decided to test that suggestion and introduced a $Concurrency parameter with a default value of 62. So in the new version the script fires up up to $Concurrency value and wait for anything to complete before starting any more processes. This improved matters greatly. Individual tests against ramdrive now run in 8 minutes and all of the 200+ db_tests run in 9 minutes with concurrency values of 8-16. About 48 is required to load a CPU on my box running against HD but that does not improve running times much.
c8e01ef98 Delete test iterators
9d50afc3b Prefix-based iterating only shows keys in prefix
14c6e1a04 Add write_stress to RocksDB Legocastle runs
db3f5e494 Update HISTORY.md
042fb053f Fix clang
2419f435a Merge pull request #816 from SherlockNoMad/GeoDBTestFix
2e4540991 Fix appveyor build failure
183cadfc8 Add OptionsSanityCheckLevel
dba5e0074 Fixed the compile error in RocksDBLite in memory_test.cc
7d7ee2b65 Add Memory Insight support to utilities
3ecbab004 Add GetAggregatedIntProperty(): returns the aggregated value from all CFs
93a966722 Merge branch 'master' of github.com:facebook/rocksdb
c9aef3c41 Add RocksDb/GeoDb Iterator interface
f31442fb5 Merge pull request #803 from SherlockNoMad/SkipFlush
dcc898b02 Merge pull request #812 from yuslepukhin/fix_windows_warnings
df7ed91ef Fix white space at end of line
a0163c068 Do not disable compiler warnings: C4101 'identifier' : unreferenced local variable C4189 'identifier' : local variable is initialized but not referenced C4100 'identifier' : unreferenced formal parameter C4296 'operator' : expression is always false
279c8e0cd Merge pull request #811 from OverlordQ/unused-variable-warning
affd83369 Fix introduced in 2ab7065 was reverted by 18285c1.
db68a2c09 Merge pull request #806 from yuslepukhin/signed_unsigned_warning
ccc8c10c0 Move skip_table_builder_flush to BlockBasedTableOption
eaaf081d1 Do not suppress C4018 'expression' : signed/unsigned mismatch The code compiles cleanly for the most part. Fix db_test. Move debug file to testutil library.
ff4499e29 Update DB::AddFile() to have less restrictions
9ac88c855 Merge branch 'master' of https://github.com/facebook/rocksdb into OptionTestFix
84992d647 Option Helper Refactoring
11c71a365 db_bench: --compaction_pri default should be rocksdb::Options().compaction_pri
335e4ce8c options_test: fix a bug of assertion
66a3a87ab Merge pull request #797 from SherlockNoMad/optionHelper
550af4ee6 Fix Travis Build Error
a6dd0831d Add Option to Skip Flushing in TableBuilder
2872e0c8c Clean and expose CreateLoggerFromOptions
296c3a1f9 "make format" in some recent commits
6388e7f4e Merge pull request #798 from yuslepukhin/readahead_buffermanagement
f4cbb90c4 Merge pull request #799 from yuslepukhin/fix_random_generator_compile
1277a48f1 Fix 80 character limit issue.
ee2c3236d Fix compilation problem on Windows. char is not a valid template parameter for std::uniform_int_distribution according to the standard. Replacing with int should be just fine.
b69b9b624 Support PlainTableOption in option_helper
c97667d9f Fix RocksDB lite build for write_stress
0d720dfc1 Use the correct variable when fetching table properties.
4b66d9534 Write stress test
47414c6cd Move include/posix/io_posix.h to util/io_posix.h
2889df84c Revert "Avoid to reply on ROCKSDB_FALLOCATE_PRESENT in include/posix/io_posix.h"
28c8758a3 Merge pull request #795 from yuslepukhin/fix_mocktable_id
5c8f2ee78 Fix MockTable ID storage On Windows two tests fail that use MockTable: flush_job_test and compaction_job_test with the following message: compaction_job_test_je.exe : Assertion failed: result.size() == 4, file c:\dev\rocksdb\rocksdb\table\mock_table.cc, line 110
72d6e758b Fix WritableFileWriter::Append() return
d0a18c284 Merge pull request #786 from aloukissas/unused_param
c37223c08 Avoid to reply on ROCKSDB_FALLOCATE_PRESENT in include/posix/io_posix.h
6fbc4f9f3 Implement smart buffer management. introduce a new DBOption random_access_max_buffer_size to limit the size of the random access buffer used for unbuffered access. Implement read ahead buffering when enabled. To that effect propagate compaction_readahead_size and the new option to the env options to make it available for the implementation. Add Hint() override so SetupForCompaction() call would call Hint() readahead can now be setup from both Hint() and EnableReadAhead() Add new option random_access_max_buffer_size support db_bench, options_helper to make it string parsable and the unit test.
d6219e4d9 Mac build break caused by include/posix/io_posix.h not declearing errno,
beb69d451 Merge pull request #765 from PraveenSinghRao/wal_filter
ab0f3b964 crash_test to trigger some less frequent crash point more frequently
7beb743cf Merge pull request #778 from Vaisman/master
4ce117c4d Merge branch 'master' into wal_filter
32cdec634 Fail recovery if filter provides more records than original and corresponding unit-test, fix naming conventions
44d4057d7 Avoid some includes in io_posix.h
2adad23a1 Fix unused parameter warnings.
b0980ff74 Fix unused parameter warnings.
bc898c5f8 Fix unused parameter warnings.
138876a62 Merge pull request #746 from ceph/wip-recycle
581f20fd8 Add LITE tests to Legocastle
3d56d868c Merge remote-tracking branch 'upstream/master'
d69111114 include/posix/io_posix.h should have a once declartion
a6962edf8 Merge pull request #783 from yuslepukhin/remove_test_conditional_compilation
3c750b59a No need to #ifdef test only code on windows
8c11c5dee Merge pull request #768 from OpenChannelSSD/to_fb_master2
6e6dd5f6f Split posix storage backend into Env and library
980a82ee2 Fix a bug in GetApproximateSizes
d0d13ebf6 fix bug in db_crashtest.py
01a41af0a Merge remote-tracking branch 'upstream/master'
5678c05d8 Use DEBUG_LEVEL=0 in make release and make clean
ac25fe6b9 Merge pull request #779 from yuslepukhin/optimize_windows_build
e154ee086 Do not build test only code and unit tests in Release builds Test code errors are currently blocking Windows Release builew We do not want spend time building in Release what we can not run We want to eliminate a source of most frequent errors when people check-in test only code which can not be built in Release. This feature will work only if you invoke msbuild against rocksdb.sln Invoking it against ALL_BUILD target will attempt to build everything.
cd3286fae Error while cmake by building from zip-archive
e3d4e1407 DBCompactionTestWithParam.ManualCompaction to verify block cache is not filled in manual compaction
033c6f1ad T7916298, bug fix
7717ad1af Adding artifacts to stress_crash CI job
0bf656b90 Don't spew warnings when flint doesn't exist
6d6776f6b Log more information for the add file with overlapping range failure
7951b9b07 make field order match initialization order
90228bb08 Merge pull request #771 from maximecaron/patch-1
2938c5c13 merge upstream changes
e3b1d23d3 Bump version to 4.2
a7b2bedfb log_{reader,write}: recyclable record format
4e07c99a9 Fix iOS build
0c59691dd Handle multiple batches in single log record - allow app to return a new batch + allow app to return corrupted record status
32c291e3c Merge branch 'master' of github.com:facebook/rocksdb into T7916298
4575de5b9 #7916298: merge tools/db_crashtest2.py into tools/db_crashtest.py
5c727de6a Merge pull request #777 from yuslepukhin/fix_win_build_uint
cfaa33f9a Update transaction iterator documentation
2f680ed09 Make index same type as auto deduced uint32_t
09f853550 uint is a not a datatype on windows.
ec1f8354a Fix the default assignment of DEBUG_LEVEL in Makefile
f18acd887 Fixed the clang compilation failure
4104e9bb6 log_reader: introduce kBadHeader; drop wal mode from ReadPhysicalRecord
9c33f64d1 log_reader: pass in WALRecoveryMode instead of bool report_eof_inconsistency
718805210 db_test_util: add recycle_log_files to set of tested options
3ac13c99d log_reader: pass log_number and optional info_log to ctor
5830c699f log_writer: pass log number and whether recycling is enabled to ctor
666376150 db_impl: recycle log files
d666225a0 db_impl: disable recycle_log_files if WAL archive is enabled
543c12ab0 options: add recycle_log_file_num option
1bcafb62f env: add ReuseWritableFile
e1a09a770 Implementation for GetPropertiesOfTablesInRange
ad471453e Allow GetProperty to report the number of currently running flushes / compactions.
277dea78f Add more kill points
a98fbacfa Moving memtable related files from util to a new directory memtable
8f143e03f Add ClearSnapshot()
f9ba79ecd crash_test to trigger fail points other than file appending more frequently
2f2de338c Run ROCKSDB_LITE tests in travis
680156ca6 crash_test to run with data sync on
e1a5ff857 Allow users to disable some kill points in db_stress
d306a7ea8 Merge pull request #773 from yuslepukhin/update_requirements
bb64d6da4 Disabling TSAN crash test
8c2fe68fd Update 4 is required for building with MS Visual Studio 13
952ad994a Fix db_test under ROCKSDB_LITE
6d730b4ae Block tests under ROCKSDB_LITE
5eee1ef2d Merge pull request #770 from Vaisman/master
63e507c59 Move ldb and sst_dump from utils to tools.
dae49e829 Make DBTest.ReadLatencyHistogramByLevel more robust
92060b215 Fix build error using Visual Studio 12
9f7413502 Error while cmake by building from zip-archive
b81b2ec25 Fix benchmarks under ROCKSDB_LITE
e587dbe03 Move manual_compaction_test.cc from util to db
666fb5df4 Remove DefaultCompactionFilterFactory.
d662b8dab Merge pull request #766 from PraveenSinghRao/lockfix
f55d3009c Make db_test_util compile under ROCKSDB_LITE
29a47cd2b Include the time unit in the comment of perf_context timers
2b925ccb4 Correct the comments in db/internal_stats.h
35ad531be Seperate InternalIterator from Iterator
91c041e57 move debug variable under ifndef
198ed5898 Merge pull request #760 from jwlent55/use-static-library-header-files
cc4d13e0a Put wal_filter under #ifndef ROCKSDB_LITE
385b41600 Merge pull request #764 from dmittendorf/fix-java-static-packaging
7062d0ea6 Make perf_context.db_mutex_lock_nanos and db_condition_wait_nanos only measures DB Mutex
1fe78a407 Fix tests failing in ROCKSDB_LITE
a6efefef7 Fix format specifiers
f7b2a7b40 Fix format specifiers
7f58ff7c3 Remove db_impl_debug from release build
1ddd91cd2 Fixed packaging of java crossbuild jar by forcing all compiled binaries to be output to the java/target directory. The uber crossbuild jar is then assembled within the java/target directory.
eb2417855 merge from master
c64ae05b1 Move TEST_NewInternalIterator to NewInternalIterator
59a0c219b Adding log filter to inspect and filter log records on recovery
0be50ed12 Merge pull request #763 from PraveenSinghRao/lockfix
a1d37602a Fixing mutex to not use unique_lock
9e819d096 Modify the way java static builds are done so that: 1) There is no need to download and install the compression libraries twice just to get access to their header files during the compile phase. 2) Ensure that the compression library headers files used during the compile phase are the same ones used to build the static library that is linked into the library.
f1fdf5205 Clean up dependency: Move db_test_util.* to db directory
237994409 Fixed an incorrect replace of const value in util/options_helper.cc
0bb8ea56b [RocksDB Options File] Add TableOptions section and support BlockBasedTable
4a7970d75 Modify the way java static builds are done so that: 1) There is no need to download and install the compression libraries twice just to get access to their header files during the compile phase. 2) Ensure that the compression library headers files used during the compile phase are the same ones used to build the static library that is linked into the library.
c4366165e Merge pull request #759 from jwlent55/statically-load-compression-libraries
fa4b5b3db Fix for the travis build caused by my previous commit
3d07b815f Passing table properties to compaction callback
64546af83 Adding parser to CI jobs
def74f876 Deferred snapshot creation in transactions
c5f3707d4 DisableIndexing() for Transactions
776bd8d5e Pass column family ID to table property collector
5a7222782 Ensure that the compression libraries are statically linked into dynamic libraries included in the Java jar. Also build the linux libraries using the portable flag to fix a problem with the linux32 build and improve the general portability of the RocksDB dynamic libraries. ==> linux32: util/crc32c.cc:318:39: error: ‘_mm_crc32_u64’ was not declared in this scope
e61d9c148 Make DBTest.AggregatedTableProperties more deterministic
b77eb16ab New Manifest format to allow customized fields in NewFile.
6732a5765 Merge pull request #756 from viveknelamangala/master
a52888ed0 Install snappy headers to standard locations using yum, so that build_tools/build_detect_platform sets -DSNAPPY flag to g++ . Current jars of rocksdb do no have snappy compression avaliable .
831101b5f Make it harder for users to run debug builds in production
000836a88 CompactionFilter::Context to contain column family ID
3a0bf873b Change RocksDB version to 4.1
77318ee14 Enable crash CI jobs
9803e0d81 compaction_filter.h cleanup
51fa7ecec Bytes read/written from cache statistics
f925208ff Create Makefile target unity_test
a065cdb38 bloom hit/miss stats for SST and memtable
40cdf797d Fix compile error on platforms without fallocate()
77e4ad7ce Fix compile failure on Travis
4049bcde3 Added boolean variable to guard fallocate() calls
aadf49fe6 Travis shouldn't fail when clang-format suggests improvements
d80ce7f99 Compaction filter on merge operands
726d9ce1a Disabling unity
026750265 Support for LevelDB SST with .ldb suffix
5855cdb6d Merge pull request #750 from yuslepukhin/fixup_build_options
7bbe10c01 Merge pull request #751 from yuslepukhin/return_noerror
e95b703b7 Mmap reads should not return error if reading past file
25c58a204 Add shared_linked DEBUG flag, remove port from among the include directories.
60b1c0a70 Fix to CI job definition
9babaeed1 Update dump_tool and undump_tool to accept Options
eb5b637fb Fix condition for bottommost level
9eaff629e Make corruption_test more robust
bf19dbff4 Fix valgrind - Initialize done variable
5c7bf56d3 [RocksDB Options] Support more options in RocksDBOptionParser for sanity check.
115427ef6 Add APIs PauseBackgroundWork() and ContinueBackgroundWork()
a39897369 Adding features to CI job description
a47bf325c Merge pull request #748 from yuslepukhin/improve_test_runs
65324a16d Improve test running scripts Introduce proper command line arguments so we can control the script Add appveyor support Add an ability to run all other (non db_tests) test executables in parallel Use .NET HashSet instead of empty valued hashtable to improve the looks TODO: Some of the tests do not use GTests and need to improve log parsing
7e4ee4231 Merge pull request #743 from edsrzf/amalgamation
7a23e4d8c New amalgamation target
e9a6808c4 Merge pull request #745 from yuslepukhin/test_appveyor_baseline
9320ffd67 Improve CI build and fix Windows build breakage Is there a way to enforce CMake additions for internal changes that seem to come w/o a PR?
03b08ba9a Return MergeInProgress when fetching from transactions or WBWI with overwrite_key
c29af48d3 Add max_file_opening_threads to db_bench
da1cf8a9b Add a missing check for deprecated options in options_helper.cc
5a51fa907 Fix accidental object copy in transactions
1e73b11af Better handling of deprecated options in RocksDBOptionsParser
a8b8295d1 Fixed a compile warning in options_test.cc under clang
74b100ac1 RocksDB Options file format and its serialization / deserialization.
75134f756 Merge pull request #741 from yuslepukhin/test_appveyor_baseline
2e7506d82 Improve CI build and build switches Add an optimized build config switch for faster test runs Change compiler options to introduce more opitmizations and be more inline with MS internal switches. Make appveyor build to utilize all the avaiable cores on the VM (parallel) Introduce new appveyor configuration for daily test runs as it would take too long to run db_test after each checkin even in paralell. With some exclusions we make it in 38 minutes. We currently fail to install ramdisk during the build. Add a powershell script to faicilitate paralell run for db_test cases.
1eff1834b Remove non-existing functions. Closes#680
16d1ba700 Clear SyncPoint Trace in DeleteSchedulerTests
30f74fa96 Make CompactionJobStatsTest.UniversalCompactionTest more robust
afe0dc539 SingleDelete support for Transactions
a263002a3 Fixed a tsan warning in db_stress.cc
e4861e7d6 Fixed a compile error in util/arena.h
0fdb4f168 Fixed a compile warning in util/arena.cc when hugetlb is not supported.
94ac8826c Merge pull request #737 from mlin/readonly-syncwal
60fa9cf0b Override DBImplReadOnly::SyncWAL() to return NotSupported. Previously, calling it caused program abort.
7df348b40 Minor fix to CI job definition
dac3f22b7 Fix the test failure
63e0f8679 Fixed a bug which causes rocksdb.flush.write.bytes stat is always zero
7ee445dd6 Fix the compile warning
174e2be5f Merge pull request #735 from jsteemann/fix-potential-leak-in-allocate
aa58958d3 prevent potential memleaks in Arena::Allocate*()
25fd743d7 Fix SingleDelete support in WriteBatchWithIndex
b6aa3f962 Fixed a memory leak issue in DBTest.UnremovableSingleDelete
7b7b5d9f1 [minor] Reuse SleepingBackgroundTask
e01f32cc3 Parameterizing email id
c58bac701 Fix valgrind failure due to memory leaks
a70d08ec0 Fix the bug of using freed memory introduced by recent plain table reader patch
628216fc1 Simplifying valgrind testing
4805fa0ea Remove ldb HexToString method's usage of sscanf
f03b5c987 Add experimental DB::AddFile() to plug sst files into empty DB
3fdb6e523 Fixed old lint errors in db/filename.cc
b349d2278 Fixed old lint errors in db/filename.h
df34aea33 PlainTableReader to support non-mmap mode
d746eaad5 RandomAccessFileReader should not inherit RandomAccessFile
03dd8f3ca Fixing punit job description
d0c31641d Internal stats WAL file synced to match meaning of the stats of the same name
48b4497f7 Merge pull request #730 from yuslepukhin/fix_write_batch_win_const_expr
a6c22e3e4 Disabling parallel test CI job
489a3e95d Re-work to support size_t max constant for 32/64-bit.
ff57c6511 [RocksJava] Fix test failure of InfoLogLevelTest
f1b9f804e Add a mode to always pick the oldest file to compact for each level
5e8f0a66d Use port::constant for std::muneric_limtis<>::max()
2754ec999 Fix Windows constexpr issue and '#ifdef' column_family_test in Release.
dd2e1eeb3 Disabling log running jobs
199744f4c Merge pull request #728 from jsteemann/fix-missing-include-header
0e65693f1 Merge pull request #727 from jsteemann/micro-optimization
4d6eb52d1 Fix to CI jobs.
3bcc072d2 Added more CI jobs
669b892f9 add missing header required for std::function
624ef456d fixed formatting. thanks @4tXJ7f for pointing me at `make format`
bbb18c827 removed unused variable of type Status, fixed indentation
470483335 pass input string to WriteBatch() by const reference
5ec129971 key_ cannot become nullptr, so no check is needed for that
834b12a8d made Size() function const because it does not modify data
1fc16cb2e Fix clang-format on Travis
e244bdf39 Merge pull request #725 from adamretter/fail-failed-java-test
7d937a090 Exit with non-zero status if one or more Java tests fail
1b598213a Check formatting in Travis
c7fba8029 Fix non-deterministic failure in backupable_db_test
014fd55ad Support for SingleDelete()
f35560d00 Merge pull request #723 from jsteemann/fix-typos
f8b770a94 fixed typos
51e1c1125 Do not flag error if file to be deleted does not exist
a5e312a7a Improving condition for bottommost level during compaction
9aca7cd6d DB::Open() to flush info log after printing DB pointer
cecd903ad Fix gflags build in Travis script
16934d495 Fix wrong constants in db_test_util
2e8e7eb39 Fix the verbosity issue in Java makefile
f21c7415a Change the log level of DB start-up log from Warn to Header.
9566342d2 Build gflags from source for Travis
3ebf11ed1 Adding the increment for a counter for a number of WAL syncs
81a61d75d Skipped tests shouldn't be failures [part 2]
1b7ea8ce8 Skipped tests shouldn't be failures
5ba3297d0 Add compaction time to log output
31a27a360 Callback for informing backup downloading added
d93a9f2aa [travis CI] Run ulimit -n 2000
2b683d497 Add DBOption.max_subcompaction to option dump
0e50a3fcc Merge issue with D46773
a7e80379b LogAndApply() should fail if the column family has been dropped
2819a1db3 Minor fix to CI job definition
df22e2fb7 Relax memory order for faster tickers
488607317 Adding Slice::difference_offset() function
5ce63e30e Merge pull request #720 from AMDmi3/fix-constant-overflow
925babc76 Merge pull request #721 from AMDmi3/printf-size_t
f171faa24 Fix printf format for size_t
4b0b0201c Fix `integer overflow in expression' error
f3170b6f6 DBImpl::FindObsoleteFiles() shouldn't release mutex between getting min_pending_output and scanning files
e467bf0de Fix valgrind error
7cb314b9e Skip some tests in ROCKSD_LITE
0bfe0573e Add gflags dependency to Travis script
5de807ac1 Add options.hard_pending_compaction_bytes_limit to stop writes if compaction lagging behind
7143242d1 Fix compaction_job_stats under ROCKSDB_LITE
592f6bf78 Merge pull request #716 from yuslepukhin/refactor_file_reader_writer_win
e2d6011f4 Minor fix to CI job definition
ad0d70ca1 Relax asserts in arena_test
03ddce9a0 Add counters for L0 stall while L0-L1 compaction is taking place
a3fc49bfd Transactions: Release Locks when rolling back to a savepoint
ddc8b4499 Address code review comments both GH and internal Fix compilation issues on GCC/CLANG Address Windows Release test build issues due to Sync
9f3a66a93 Improvements to CI jobs
7db1471cc Minor fix to CI job
c67d20689 Fixed arena_test failure due to malloc_usable_size()
34cedaff6 Initialize variable to avoid warning
aeb461268 Add counters for seek/next/prev
45e9e4f0b Refactor NewTableReader to accept TableReaderOptions
ddb950f83 Fixed bug in compaction iterator
30e82d5c4 Refactor to support file_reader_writer on Windows. Summary. A change https://reviews.facebook.net/differential/diff/224721/ Has attempted to move common functionality out of platform dependent code to a new facility called file_reader_writer. This includes: - perf counters - Buffering - RateLimiting
af7cdbf64 Run full test suite in Travis
c25f6a85b Removed __unused__ attribute
6db0a939d Fix DBCompactionTest failure with parallel L0-L1 compactions
8aa1f1519 Refactored common code of Builder/CompactionJob out into a CompactionIterator
41bce0586 CI job improvements
95ffc5d2b Correct ASSERT_OK() in ReadDroppedColumnFamily
3c37b3ccc Determine boundaries of subcompactions
112664408 Relaxing consistency detection to include errors while inserting to memtable as WAL recovery error.
abc7f5fdb Make DBTest.ReadLatencyHistogramByLevel more robust
ac9bcb55c Set max_open_files based on ulimit
4cbd2f9aa Merge pull request #714 from facebook/travisformac
d0df54d1f Run travis tests on OS X
a55e5a52a Merge pull request #711 from facebook/testtravis
2b676d5bb Upgrade travis to new architecture
c66d53fee Fixed minor issue in CompressionTypeSupported()
f3f2032c4 Release RocksDB 4.0.0
44b6e99e1 update max_write_buffer_number_to_maintain docblock
aa6eed0c1 Transaction stats
25dbc579f Update HISTORY file for transactions
52386a1e7 Minor fix to sandcastle jobs commands
b5b2b75e5 better tuning of arena block size
342ba8089 Make DBTest.OptimizeFiltersForHits more deterministic
e17e92ea1 Relaxed assert in forward iterator
5e94f68f3 TransactionDB Custom Locking API
0ccf2db38 Fixed broken build due to format specifier
6bdc484fd Added Equal method to Comparator interface
7a31960ee Tests for ManifestDumpCommand and ListColumnFamiliesCommand
778cf4449 Adding email notification.
3a0df7f16 Fixed comparison in ForwardIterator when computing hint for GetNextLevelIndex()
91f3c9079 Fix case when forward iterator misses a new update
ff1953c89 Merge pull request #707 from dkorolev/master
7b463e657 Fixed a typo in INSTALL.md
d9f42aa60 Adding a verifyBackup method to BackupEngine
50dc5f0c5 Replace BackupRateLimiter with GenericRateLimiter
20ef64cae Moving jobs to use gcc-4.9
0f1aab6c1 Add SetLockTimeout for Transactions
14456aea5 Fix compile
76f286cc8 Optimize bloom filter cache misses
0e6e5472e Fixed a compile warning in rocksjni/loggerjnicallback.cc
b8a962d4f Adding commands for few more CI jobs.
8a2d59a35 Add Cloudera's blog post to USERS.md
3c9cef1ee Unified maps with Comparator for sorting, other cleanup
3e0a672c5 Bug fix: table readers created by TableCache::Get() doesn't have latency histogram reported
b42cd6bed Remove the need for LATEST_BACKUP in BackupEngine
0f763db20 Merge pull request #705 from yuslepukhin/rate_limiter_fix
20c44fefb t6913679: Use fallocate on LOG FILESS
f14c3363e Make WinEnv::NowMicros return system time Previous change for the function 555ca3e7b7 (diff-bdc04e0404c2db4fd3ac5118a63eaa4a) made use of the QueryPerformanceCounter to return microseconds values that do not repeat as std::chrono::system_clock returned values that made auto_roll_logger_test fail.
aad0572f8 Fixed the build issue of rocksdbjavastaticrelease
5508122ed Fix a perf regression in ForwardIterator
b72200777 Fix listener_test when using ROCKSDB_MALLOC_USABLE_SIZE
40cd91b7f Fixed compile warning in rocksdbjava
90415cfeb Fixed a compile warning in linux32 environment.
9d6503f88 Fix arena_test test break using glibc-2.17
77a28615e Support static Status messages
18db1e469 better db_bench options for transactions
0be260523 Merge pull request #702 from PraveenSinghRao/remove_spurious
8b689546b Add Subcompactions to Universal Compaction Unit Tests
c6d870ffb Merge branch 'arcpatch-D45741'
57b3a8773 Adding sandcastle determinator for RocksDB
3d78eb66b Arena usage to be calculated using malloc_usable_size()
effd9dd1e Fix deadlock in WAL sync
64f07deb8 remove spurious compression definitions
72a9b73c9 Removed unnecessary checks in DBTest.ApproximateMemoryUsage
cb164bfc4 Do not delete iterators for immutable memtables.
7a0dbdf3a Add ZSTD (not final format) compression type
e2db15efd Merge pull request #701 from PraveenSinghRao/usewinapi_notcruntime
e853191c1 Fix DBTest.ApproximateMemoryUsage
7c916a5d3 Merge pull request #699 from OpenChannelSSD/to_fb_master
0886f4f66 Helper functions to support direct IO
7e327980a Remove usage of C runtime API that has file handle limitation
8ef0144e2 Add argument --show_table_properties to db_bench
1fb2abae2 ColumnFamilyOptions serialization / deserialization.
5f4166c90 ReadaheadRandomAccessFile -- userspace readahead
16ebe3a2a Mmap reads should not return error if reading past file
d286b5df9 DBIter to out extra keys with higher sequence numbers when changing direction from forward to backward
3795449c9 Fix DBTest.GetProperty
a7834a129 Merge pull request #698 from yuslepukhin/address_noexcept_windows
9ccf1bd3e Correct the comment for GetProperty() API.
fbe2c05f5 s/NOEXCEPT/ROCKSDB_NOEXCEPT
6924d7582 Address noexcept and const integer lambda capture VS 2013 does not support noexcept. Complains about usage of ineteger constant within lambda requiring explicit capture.
2f8d71ec0 Moving sequence number compaction variables from SubCompactionState to CompactionJob
bab9934d9 Fix build failure caused by bad merge.
4d28a7d8a Add a whitebox test for deleted file iterators.
249fb4f88 Fix use of deleted file iterators with incomplete iterators
53b88784d Add throttling to multi-threaded backups
09d982f9e Fix compact_files_example
6996de87a Expose per-level aggregated table properties via GetProperty()
86d6c3cde Fix Windows build
20d1e547d Common base class for transactions
205083297 Fixing race condition in DBTest.DynamicMemtableOptions
e46bcc08b Remove an extra 's' from cur-size-all-mem-tabless
4ab26c5ad Smarter purging during flush
4c81ac0c5 Fix benchmark report script
b6def58f7 Changed 'num_subcompactions' to the more accurate 'max_subcompactions'
c85296846 db_iter_test: add more test cases for the data race bug
9130873a1 Add options.new_table_reader_for_compaction_inputs
07d2d3416 Add a counter about estimated pending compaction bytes
41a0e2811 Improve defaults for benchmarks
a203b913c Fixed a rare deadlock in DBTest.ThreadStatusFlush
962aa6429 Merge pull request #695 from yuslepukhin/address_windows_build
5bf890762 More indent adjustment.
e2a9f43d6 Adjust indent
6e9a260b0 Merge branch 'address_windows_build' of https://github.com/yuslepukhin/rocksdb into address_windows_build
1cac89c9b Address windows build issues Intro SubCompactionState move functionality =delete copy functionality #ifdef SyncPoint in tests for Windows Release builds
f25f06ddd Address windows build issues Intro SubCompactionState move functionality =delete copy functionality #ifdef SyncPoint in tests for Windows Release builds
027ca5b2c Total SST files size DB Property
b604d2562 Removing unused variables to fix build
1b114eed4 Free file iterators for files which are above the iterate upper bound to Improve memory utilization
3fd70b05b Rate limit deletes issued by DestroyDB
df79eafcb Introduce GetIntProperty("rocksdb.size-all-mem-tables")
888fbdc88 Remove the contstaint that iterator upper bound needs to be within a prefix
137c37667 Removing variables used only in assertions to prevent build error
b47cc5851 Bounding Number of Subcompactions
e58e1b18e Make tailing iterator show new entries in memtable.
9ec957159 DBOptions serialization and deserialization
b2df20a89 Make HashCuckooRep::ApproximateMemoryUsage() return reasonable estimation.
601b1aaca Fixing Failed Assertion in Subcompaction State Diff
f0da6977a [Parallel L0-L1 Compaction Prep]: Giving Subcompactions Their Own State
f32a57209 Simplify querying of merge results
72613657f Measure file read latency histogram per level
b7198c3af reduce db mutex contention for write batch groups
603b6da8b Add options.compaction_measure_io_stats to print write I/O stats in compactions
dc9d5634f Change master to 3.14
b78c8e07d Merge pull request #689 from msb-at-yahoo/add-tools-target
9f0dd2229 Add a 'tools' target.
463720712 Add test case to repro the mispositional iterator in a low-chance data race case
3bd9db420 [Cleanup] Remove RandomRWFile
c3466eab0 Have Transactions use WriteBatch::RollbackToSavePoint
0db807ec2 Transaction error statuses
c2f2cb021 Pessimistic Transactions
c2868cbc5 Use manual_compaction for compaction_job_test
6b2d57039 Fix Windows build by adding snapshot_impl to CMakeLists
e61fafbe7 Fixed clang-build error in util/thread_local.cc
cee1e8a08 Parallelize LoadTableHandlers
4249f159d Removing duplicate code in db_bench/db_stress, fixing typos
a03085b55 Fix linters on non-fb machines
1ae27113c reduce comparisons by skiplist
b47d65b31 Fixed Segmentation Fault in db_stress on OSX.
a1581eca8 Modernize RocksDB linters
a9dcc0a63 Fix clang build
2cf0f4f47 Adding wal_recovery_mode log message
68f934355 Better CompactionJob testing
22dcaaff3 More accurate time measurement for delete_scheduler_test
0a7ea582c Add auto-build manifest for appveyor
0093271ee Merge pull request #685 from flandr/fix-tls-build
ac04a6cfb Fix OSX + Windows build
16ea1c7d1 simple ManagedSnapshot wrapper
257ee895f Fixed memory leaks
254c4fb88 In HISTORY.md Switch unreleased notes to 3.13
40f893f4a Fix delete_scheduler_test valgrind error
6a4aaadcd Avoid type unique_ptr in LogWriterNumber::writer for Windows build break
d7314ba75 Fixing endless loop if seeking to end of key with seq num 0
48e6e9aa8 Add util/delete_scheduler_impl.cc to CMakeLists.txt
c7742452e Add Statistics.getHistogramString() to print more detailed outputs of a histogram
29b028b0e Make DeleteScheduler tests more reliable
fca88f8e1 valgrind_check to exit on test failures
7d364d0d9 Fix build failure
960d936e8 Add function 'GetInfoLogList()'
7ccd1c80a Add two unit tests for SyncWAL()
3ae386eaf Add statistic histogram "rocksdb.sst.read.micros"
8ecb51a7e "make commit-prereq" should clean up rocksjava properly
9aec75fbb Enable DBTest.FlushSchedule under TSAN
bd2fc5f5f Fix TSAN for delete_scheduler_test
8e01bd114 Fix misplaced position for reversing iterator direction while current key is a merge
c46507102 Removing duplicate code
e06cf1a09 [wal changes 3/3] method in DB to sync WAL without blocking writers
5dc3e6881 Update Tests To Enable Subcompactions
c45a57b41 Support delete rate limiting
102ac118b Update JAVA-HISTORY.md for v3.13
3a1d4e6c9 Merge pull request #670 from skunkwerks/fix_osx_shared_library_names
f5d072ae6 Fixed RocksJava test failure of shouldSetTestCappedPrefixExtractor
f39cbcb0a Merge pull request #654 from adamretter/remove-emptyvalue-compactionfilter
f0b5bcc7b add support for capped prefix extractor in java
18ba58a94 Upgrading jemalloc from 3.6.0 to the latest for fbcode+gcc 4.8.1
ce21afd20 Expose the BackupEngine from the Java API
b0d12a135 Merge pull request #569 from adamretter/travis-java-api
241bb2aef Make DBCompactionTest.SkipStatsUpdateTest more stable.
3424eeb1e Polish HISTORY.md
cf3e05304 crash_test cleans up directory before testing if TEST_TMPDIR is set
24daff6d7 Fix a typo and update HISTORY.md for NewCompactOnDeletionCollectorFactory().
14d0bfa42 Add DBOptions::skip_sats_update_on_db_open
e2a3bfe74 First half of whitebox_crash_test to keep crashing the same DB
2e73bd4ff crash_test to put DB under TEST_TMPDIR
1205bdbce crash_test to cover simply cases
d5c0a6da6 Merge branch 'master' of github.com:facebook/rocksdb
2d41403f4 Made change to fix the memory leak
92f7039ee fix memory corruption issue in sst_dump --show_compression_sizes
be8621ffa Fix compile warning in compact_on_deletion_collector in some environment
26894303c Add CompactOnDeletionCollector in utilities/table_properties_collectors.
20b244fcc Fix CompactFiles by adding all necessary files
87df6295d Make SuggestCompactRangeNoTwoLevel0Compactions deterministic
40c64434d Parallelize L0-L1 Compaction: Restructure Compaction Job
47316c2d0 dump_manifest supports DB with more number of levels
bd852bf11 Fixed typos in db_stress
544be638a Fixing fprintf of non string literal
193dc977e Fixing dead code in table_properties_collector_test
05d4265a2 Merge branch 'master' of github.com:facebook/rocksdb
4be6d4416 Compression sizes option for sst_dump_tool
8161bdb5a WriteBatch Save Points
7bfae3a72 tools/db_crashtest2.py should run on the same DB
d06c82e47 Further cleanup of CompactionJob and MergeHelper
e95c59cd2 Count number of corrupt keys during compaction
221a94a5f Another attempt at adding the Java API and tests to the travis build
1bdfcef7b Fix when output level is 0 of universal compaction with trivial move
6a82fba75 Add missing hashCode() implementation
f73c80143 Fixing Java tests.
14f413760 Correct the comment of DB::GetApproximateSizes
8279d4197 Merge pull request #667 from yuslepukhin/fix_now_microsec_win
eb8e3b4c7 Fix shared library names on OSX
555ca3e7b Fix WinEnv::NowMicrosec * std::chrono does not provide enough granularity for microsecs and periodically emits duplicates * the bug is manifested in log rotation logic where we get duplicate log file names and loose previous log content * msvc does not imlement COW on std::strings adjusted the test to use refs in the loops as auto does not retain ref info * adjust auto_log rotation test with Windows specific command to remove a folder. The test previously worked because we have unix utils installed in house but this may not be the case for everyone.
82f148ef9 Fix test DBCompactionTest.PartialCompactionFailure undeterministic failure
6002801e0 Abandon ROCKSDB_WARNING
4922af6f8 fixed DBTest.GetPropertiesOfAllTablesTest and DBTest.GetUserDefinedTablaProperties flakiness
3bf9f9a83 cleaned up PosixMmapFile a little
fe09a6dae [wal changes 2/3] write with sync=true syncs previous unsynced wals to prevent illegal data loss
06aebca59 Report live data size estimate
85ac65536 Tests to avoid to use TMPDIR directly
f09d45690 Merge pull request #664 from yuslepukhin/add_tests_fix_sanity
31b35c902 Add missing tests, fix db_sanity Add heap_test, merge_helper_test Fix uninitialized pointers in db_sanity_test that cause SIGSEV when DB::Open fails in case compression is not linked.
66a3cbc54 Merge pull request #663 from yuslepukhin/fix_windows_build_refactor
ac5e441ad Fix windows build after refactoring Missing and duplicate files in CMake Missing definition of port::Crash
02b635fa3 Fix undeterministic failure of DBTest.GetPropertiesOfAllTablesTest
3dbf4ba22 RangeSync not to sync last 1MB of the file
7219088cd Move general compaction tests from db_test.cc to db_compaction_test.cc
0adecd9f4 Add db_inplace_update_test back to Makefile
064294081 Improved FileExists API
9f1de9518 Revert Makefile
6867fb19c Revert "Add missing db_log_iter_test in the test list"
443c6646b Move remaining universal compaction tests from db_test.cc to db_universal_compaction_test.cc
7462286d3 Move in-place-update related tests from db_test.cc to db_inplace_update_test.cc
03467bdd4 Add missing db_log_iter_test in the test list
331954ab8 Fixed DBTestUniversalManualCompactionOutputPathId test
a75f23eb8 Relax assertions in unit DropWrites to be more permissible
ee80432ff db_bench add an option of --universal_allow_trivial_move
58b4209e0 Trigger non-trivial compaction in fault_injection_test
59eca2cc9 Make memenv_test runnable in ROCKSDB_LITE
aa8ac6445 Skip unsupported tests in ROCKSDB_LITE
ce9712d34 Make mock_env_test runnable in ROCKSDB_LITE
c06d1d839 Make merge_test runnable in ROCKSDB_LITE
144d2910d Block backupable_db_test_lite in ROCKSDB_LITE
0d1d9aeeb Block plain_table_db_test in ROCKSDB_LITE
4853e228e Make table_test runnable in ROCKSDB_LITE
f0fe9126f Fix compile for write_callback_test in ROCKSDB_LITE
cf6a7bebc Block cuckoo table tests in ROCKSDB_LITE
20922c4a5 Make compaction_picker_test runnable in ROCKSDB_LITE
6e9fbeb27 Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env
5ec829bc4 Cleaning up CYGWIN define of fread_unlocked to port
26ca89319 Block document_db_test in ROCKSDB_LITE
35ca59364 Don't let flushes preempt compactions
79373c372 Fix ROCKSDB_WARNING
74c755c55 Added JSON manifest dump option to ldb command
a96fcd09b Deprecate CompactionFilterV2
1d20fa9d0 Fixed and simplified merge_helper
aede5cd8e Merge pull request #656 from qinzuoyan/fb-master
d730c3677 Merge pull request #657 from yuslepukhin/ensure_clean_public_headers
ac2b9367f Fix a typo in variable
415c47323 Merge after rebasing
d1a457181 Ensure Windows build w/o port/port.h in public headers
6c0c8dee7 Fix data loss after DB recovery by not allowing flush/compaction to be scheduled until DB opened
e4af3bfb2 Test for compaction of corrupted keys
91bf1b80e Java facility to use the RemoveEmptyValueCompactionFilter
3d00271e4 The ability to specify a compaction filter via the Java API
62dec0e2b RemoveEmptyValueCompactionFilter - A compaction filter which removes entries which have an empty value
c5bca5319 Fix compile on Mac
487bba434 extend temp str buffer size
247690fe3 Ensure Windows build w/o port/port.h in public headers
84c3577af fix append bug in DumpDBFileSummary()
43e982562 Fix mongo build -take 2
d8263d958 Unbreak mongo build
12c5528a8 Bump to RocksDB 3.13
81d072623 move convenience.h out of utilities
beb19ad0d Fixing delete files in Trivial move of universal compaction
c61396069 Build fix.
2c8de0eca Update --help message in db_bench.
6b2d44b2f Refactoring of writing key/value pairs
e1c99e10c Replace std::priority_queue in MergingIterator with custom heap, take 2
9a6a0bd8c Style fix in compaction_job.cc
e94c510c3 Make ldb_test not depend on compression
ddad40e93 Fixed nullptr deref and added assert
1bc8eb877 make coverage should execute sequentially
801df912a Move UniversalCompaction related db-tests to db_universal_compaction_test.cc
3ca6b2541 Move TailingIterator tests from db_test.cc to db_test_tailing_iterator.cc
ce829c77e Make TransactionLogIterator related tests from db_test.cc to db_log_iter_test.cc
c3f98bb89 Move CompactionFilter tests in db_test.cc to db_compaction_filter_test.cc
0936362a7 Block SyncPoint in util/db_test_util.h in released Windows mode.
05e194158 Merge pull request #639 from cleaton/setMaxTableFileSize
fc2b71d9c Merge pull request #655 from adamretter/java-make-resolve-maven
8a9fca261 Better error handling in BackupEngine
18d5e1bf8 Remove db_impl_readonly dependency on utilities
9d22a9737 Resolve Java test dependencies from local maven repo if present
49640bd82 Allow write_batch_test to run with ROCKSDB_LITE
a9c510951 Deprecate purge_redundant_kvs_while_flush
5aea98ddd Deprecate WriteOptions::timeout_hint_us
ae29495e4 Avoid manipulating const char* arrays
ab137af4b Partial cleanup of CompactionJob
1879d9370 Add ldb_test.py back to `make check`
b10cf4e2e Move DynamicLevel related db-tests to db_dynamic_level_test.cc
e290f5d3c Block reduce_levels_test in ROCKSDB_LITE
04d201fa0 Block spatial_db_test in ROCKSDB_LITE
49f42ad03 Move global static functions in db_test_util to DBTestBase
625467a08 Move reusable part of db_test.cc to util/db_test_util.h
e8e8c9049 fix compile for optimistic_transaction_test under ROCKSDB_LITE
8bca83e5d Add tombstone information in CompactionJobStats
f9728640f "make format" against last 10 commits
76d3cd328 Fix public API dependency on internal codes and dependency on MAX_INT32
5fd11853c Print Fast CRC32 support information in DB LOG
a6e38fd17 Fix a uncleaned counter in PerfContext::Reset()
e41cbd9c2 Merge pull request #646 from yuslepukhin/ms_win_port
4cab5ebec Merge branch 'ms_win_port' of https://github.com/yuslepukhin/rocksdb into ms_win_port
296de4ae6 Address review comments Rule of five: add destructor Add a note to COMMIT.md for 3rd party json.
041b6f95a perf_context: report time spent on reading index and bloom blocks
d08ba9f0c Merge branch 'ms_win_port' of http://vstfbing:8080/tfs/Bing/_git/repo.RocksDB into ms_win_port
805fe84ba Conditional use of third-party libraries Committed by Alexander Zinoviev <alexander.zinoviev@me.com> 7/9/2015 2:42:41 PM
5555cc500 Improve build system
c903ccc4c Merge from github/master
54d124a38 Conditional use of third-party libraries
7189e90c2 Fix a noisy unit test.
1f4d56570 Add db_bench flag to set cache_index_and_filter_blocks
5c7913233 Revert the changes related to Options, as requested to seperate them into a different patch.
d8586ab22 All of these are in the new code added past 3.10 1) Crash in env_win.cc that prevented db_test run to completion and some new tests 2) Fix new corruption tests in DBTest by allowing a shared trunction of files. Note that this is generally needed ONLY for tests. 3) Close database so WAL is closed prior to inducing corruption similar to what we did within Corruption tests.
4bed00a44 Fix function name format according to google style
e2e3d84b2 Added multi WAL log testing to recovery tests.
5219226d3 Merge branch 'ms_win_port' of http://vstfbing:8080/tfs/Bing/_git/repo.RocksDB into ms_win_port
ef4b87f1b Commit both PR and internal code review changes
95f4c2bcb Conditional use of 3rd-party libraries
4f56632b1 Fix occasional failure in compaction_job_test
411c8e3d1 Build fail fix
b7a2369fb Revert "Replace std::priority_queue in MergingIterator with custom heap"
c0b23dd5b Enabling trivial move in universal compaction
d8e3e766f Fixed a bug in test ThreadStatusSingleCompaction
57d216ea6 Remove assert(current_ == CurrentReverse()) in MergingIterator::Prev()
59b50dcef Update HISTORY.md for Listener
4ce5be425 fixed leaking log::Writers
685582a0b Revert two diffs related to DBIter::FindPrevUserKey()
e12b40399 Initialize threads later in constructor
58d7ab3c6 Added tests for ExpandWhileOverlapping()
155ce60da Fix compaction_job_test
b6655a679 Replace std::priority_queue in MergingIterator with custom heap
e25ee32e3 Arena needs mman header for mmap
d2f0912bd Merge the latest changes from github/master
35cd75c37 Introduce InfoLogLevel::HEADER_LEVEL
acee2b08a Fixed endless loop in DBIter::FindPrevUserKey()
218487d8d [wal changes 1/3] fixed unbounded wal growth in some workloads
feb99c31a Merge remote-tracking branch 'origin' into ms_win_port
e70115e71 Fix unity build by removing anonymous namespace
4159f5b87 Prepare 3.12
a69bc91e3 Multithreaded backup and restore in BackupEngineImpl
9dbde7277 Merge remote-tracking branch 'origin' into ms_win_port
03d433ee6 [RocksJava] Fixed test failures
326da912d Add string.h to Histogram as we init the array out of curly braces
ca2fe2c1b Address GCC compilation issues
19e13a595 Fix header inclusion
18285c1e2 Windows Port from Microsoft
c00948d5e [RocksJava] Fix test failure of compactRangeToLevel
05e283196 Allocate LevelFileIteratorState and LevelFileNumIterator from DB iterator's arena
436ed904d Add rpath option to production builds for 4.8.1 toolchain
b0f1927db Increasing timeout for drop writes.
ec70fea4c Fix a comparison in DBIter::FindPrevUserKey()
501591c42 Make column_family_test runnable in ROCKSDB_LITE
91cb82f34 Merge branch 'master' of github.com:facebook/rocksdb
09f5a4b48 set -e in fb_compile_mongo.sh
6199cba99 Fix race in unit test.
0a019d74a Use malloc_usable_size() for accounting block cache size
4cbc4e6f8 Call merge operators with empty values
619167ee6 Fix mac compile
472e64d39 Improve fb_compile_mongo.sh
c9cd404bc Make flush check for shutdown
4fb09c687 Updating SeekToLast with upper bound
dadc42976 Reproducible MongoRocks compile with FB toolchain
62a8fd154 Make stringappend_test runnable in ROCKSDB_LITE
48da7a9ca Improve the comment for BYTES_READ in statistics.
72cab8895 Block redis_test in ROCKSDB_LITE
dec2c9f56 Make table_properties_collector_test runnable in ROCKSDB_LITE
0b1ffe2e1 Remove -Wl,--no-as-needed flag when making shared_lib in OSX and IOS
674b1181c Bottommost level compaction option
782a1590f Implement a table-level row cache
de85e4cad Introduce WAL recovery consistency levels
530534fce Fix trivial move merge
7015fd81c Add read_nanos to IOStatsContext.
7160f5d80 Fix broken gflags link
dda74111a add setMaxTableFilesSize Options unit test
d62b6ed83 add setMaxTableFilesSize to JNI interface
e1d3c7dbe Fixing valgrind error in checkpoint_test
3bdec09cb Remove ldb_tests.py from make check until it is working again.
15325bf55 First version of rocksdb_dump and rocksdb_undump.
04251e1e3 Add wal files to Checkpoint for multiple column families.
18cc5018b Fix memory leaks in PinnedUsageTest
bf03f59c1 Disable CompressLevelCompaction() if Zlib is not supported
df719d496 Make autovector_test runnable in ROCKSDB_LITE
4d6d47688 Block geodb_test in ROCKSDB_LITE
71b438c4a Remove unused target --- compactor_test
eade498bd Block utilities/write_batch_with_index in ROCKSDB_LITE
760e9a94d Fail DB::Open() when the requested compression is not available
69bb210d5 Add Cache.GetPinnedUsageUsage()
4eabbdb7e Skip bottommost level compaction if possible
4b8bb62f0 Don't dump DBOptions for each column family
176f0bedc Merge branch 'master' of github.com:facebook/rocksdb
bb1c74ce1 Fixed a bug of CompactionStats in multi-level universal compaction case
a66b8157d Merge branch 'master' of github.com:facebook/rocksdb
f06be62fd Replace %llu with format macros in ParsedInternalKey::DebugString())
2dc3910b5 Add --benchmark_write_rate_limit option to db_bench
12e030a99 Use CompactRangeOptions for CompactRange
c89369f57 Move dockerbuild.sh to build_tools/
4716ab4d1 Merge pull request #638 from HolodovAlexander/master
25d600569 Clean up InstallSuperVersion
1369f015e Only initialize the ThreadStatusData when necessary.
1a08d0beb Block c_test in ROCKSDB_LITE
40f562e74 Allow GetApproximateSize() to include mem table size if it is skip list memtable
d59d90bb1 db_bench periodically writes QPS to CSV file
46296cc86 Cygwin build not to use -fPIC
bee8d033f Removed two unused macros in iostats_context
5fec96387 Fixed false alarm of size comparison in compaction_job_stats_test
cccd2199a Revert skip bottommost compaction
20f2b5425 Skip bottom most level compaction if no compaction filter
7842920be Slow down writes by bytes written
a84df655f Don't let two L0->L1 compactions run in parallel
d6ce0f7c6 Add largest sequence to FlushJobInfo
ab455ce49 fix clang build
3eddd1abe Add Env::GetThreadID(), which returns the ID of the current thread.
73faa3d41 Handling edge cases for ReFitLevel
bffaf0a8b Merge pull request #631 from mkhq/patch-1
821cff114 Re-generate WriteEntry on WBWIIterator::Entry()
8b7be1808 Updated OS X instructions, replace homebrew/dupes with homebrew/versions
d03f11090 Link all libraries when building shared libraries
75222d130 Revert "Fix compile"
47f1e7212 Merge pull request #630 from rdallman/c-wb-logdata
51440f83e Fix compile
4949ef08d Re-generate WriteEntry on WBWIIterator::Entry()
735df6655 C: add WriteBatch.PutLogData support
e409d3d74 Make "make all" work for CYGWIN
62c3a9579 Add test for iteration+mutation of WBWI
d9b3338eb Add Yahoo's blog post about Sherpa to USERS.md
75d7075a8 Print info message about files need compaction for debuging purpose
406a5682e Fix hang when closing a DB after doing loads with WAL disabled.
d8c8f08c1 GetSnapshot() and ReleaseSnapshot() to move new and free out of DB mutex
643bbbf08 Use nullptr for default compaction_filter_factory
21f2492ac Fix CYGWin release build
f02ce0c65 Fix ASAN errors in c_test
133130a4f Merge pull request #625 from rdallman/c-slice-parts-support
de4d172d0 Merge pull request #622 from rdallman/c-multiget
6df589b44 Add TablePropertiesCollector::NeedCompact() to suggest DB to further compact output files
2e764f06e [API Change] Improve EventListener::OnFlushCompleted interface
7322c7401 Revert incorrect commit
31e60e2a7 Unlock mutex in ReFitLevel
7647df8f9 Fixed the tsan failure in util/compaction_job_stats_impl.cc
b2785472c Fix compile
3ce3bb3da Allowing L0 -> L1 trivial move on sorted data
bb808eadd Changed the CompactionJobStats::output_key_prefix type from char[] to string.
0b3172d07 Add EventListener::OnTableFileDeletion()
211a195d4 C: add MultiGet support
5dc174e11 C: add support for WriteBatch SliceParts params
2d0b9e5f0 Fix compile on darwin
3af668ed1 Fix DBTest.MigrateToDynamicLevelMaxBytesBase slowness with valgrind
408cc4b8e Revert "Merge pull request #621 from rdallman/c-slice-parts-support"
78382d4ba Merge pull request #621 from rdallman/c-slice-parts-support
ca8b85ac0 better document max_write_buffer_number_to_maintain
0483dab2a Remove a TODO that has been done
8afafc278 Fix compile warning in db/db_impl
fe5c6321c Allow EventListener::OnCompactionCompleted to return CompactionJobStats.
3083ed212 Fixed heap-use-after-free error in compaction_job_test.cc
8d8d4e45b Fixed ROCKSDB_LITE compile error due to the missing of TableFileCreationInfo
ab946af08 Fix a compile warning in listener_test.cc
fc8382127 Add EventListener::OnTableFileCreated()
898e803fc Add a stats counter for DB_WRITE back which was mistakenly removed.
ac81130fa Fix Bug: CompactRange() doesn't change to correct level caused by using wrong level
ec7a94436 more times in perf_context and iostats_context
4266d4fd9 Allow users to migrate to options.level_compaction_dynamic_level_bytes=true using CompactRange()
d333820ba Removed DBImpl::notifying_events_
495ce6018 Fixed compile warning in compact_files_example.cc
2ecac9f96 add rocksdb::WritableFileWrapper similar to rocksdb::EnvWrapper
a187e66ad Merge pull request #617 from rdallman/wb-merge-sliceparts
16c197627 Fixed db_stress
4c181f08b Fix compile on darwin
bc7a7a400 fix LITE build
832271f6b Fixed a compile warning in db_stress in NDEBUG mode.
dc9d70de6 Optimistic Transactions
d5a0c0e69 Fixed a compile warning in db_stress
ebfdb3c7f Fixed a compile error in ROCKSDB_LITE
9ffc8ba02 Include EventListener in stress test.
a3da59022 Decrease number of jobs in make release
21cd6b7ad C: add support for WriteBatch SliceParts params
a0635ba3f WriteBatch.Merge w/ SliceParts support
c81535103 Support saving history in memtable_list
ec4ff4e99 Rename EventLoggerHelpers EventHelpers
672dda9b3 [API Change] Move listeners from ColumnFamilyOptions to DBOptions
3ab8ffd4d Compaction now conditionally boosts the size of deletion entries.
a81ac2412 Merge pull request #615 from rdallman/master
6d299b70b Fixed a bug in EventLoggerHelpers::LogTableFileCreation
a0580205c Removed an unused private variable in db_impl.h
328ad902a update an import path to fit in with the rest of the kids
9c38ce1d0 C: extra bbto / noop slice transform
8d26799fe Merge pull request #614 from arschles/docker
32198343f fix typo in c_simple_example
6116ccc23 moving dockerfile to root
d90cee9fd adding docker build script and dockerfile
ea6d3a8ac Don't skip last level when calculating compaction stats
5c224d1b7 Fixed two bugs on logging file deletion.
dc81efe41 Change the log-level of DB summary and options from INFO_LEVEL to WARN_LEVEL
687214f87 Ensure ColumnFamilyOptions.num_levels >= 2 when level compaction is used.
2abb59268 Avoid logging under mutex in DBImpl::WriteLevel0TableForRecovery().
309a9d076 Run tests sequentally if J=1
7fee8775a Allow EventLogger to directly log from a JSONWriter.
7a3577519 Don't artificially inflate L0 score
4cb4d546c Set stats_dump_period_sec to 600 by default
e2c1d4b57 [Public API Change] Make DB::GetDbIdentity() be const function.
eaf61ba9f Minor text correction
f16c0b289 Merge pull request #613 from DerekSchenk/DerekSchenk-patch-issue-606
d1a978ae3 Rename JSONWritter to JSONWriter
3e0817541 Add LDFLAGS to Java static library
812c461c9 Dump db stats in WARN level
b588505a7 Update HISTORY.md for GetThreadList() update.
944043d68 Add --wal_bytes_per_sync for db_bench and more IO stats
d5de04d20 Update history for 3.11
08b6b3796 FORCE_GIT_SHA
04feaeebb Fix comparison between signed and usigned integers
4a855c079 Add an option wal_bytes_per_sync to control sync_file_range for WAL files
b0fdda4ff Allow flushes to run in parallel with manual compaction
74f3832d8 Fixed compile errors due to some gcc does not have std::map::emplace
0c8017dba Remove duplicated code
fb5bdbf98 DBTest.DynamicLevelMaxBytesCompactRange: make sure L0 is not empty before running compact range
6fa708512 CompactRange skips levels 1 to base_level -1 for dynamic level base size
84a9c6a53 add comment
eeb44366b C api: human-readable statistics
3f0867c0f Allow GetThreadList to report Flush properties.
a66f643e9 Use a better way to initialize ThreadStatus::kNumOperationProperties.
7413306d9 Take a chance on a random file when choosing compaction
8c52788f0 Use version defined in Makefile in rocksdb_build_git_sha
5aad88129 DBTest.DynamicLevelMaxBytesBase2: remove an unnecesary check
ec43a8b9f Universal Compaction with multiple levels won't allocate up to output size
714fcc067 Make ThreadStatus::InterpretOperationProperties take const uint64_t*
bc68bd5a1 db_bench to support rate limiter
df1f87a88 Fixed compile error in db/column_family.cc
14431e971 Fixed a bug in EventListener::OnCompactionCompleted().
dbd95b753 Add more table properties to EventLogger
b5881762b Reset parent_index and base_index when picking files marked for compaction
711465cce API to fetch from both a WriteBatchWithIndex and the db
3996fff8a Fix clang build - add override
d97813906 SuggestCompactRange() is manual compaction
2fe24e4b5 Don't treat warnings as error when building rocksdbjavastatic
beda81c53 rm -rf in make clean
50eab9cf3 Fix BackupEngine
962f8ba33 Bump to 3.11
37bbd3185 Merge pull request #601 from adamretter/jdb-bench
77a5a543a Allow GetThreadList() to report basic compaction operation properties.
65fe1cfbb Cleanup CompactionJob
df4130ad8 fix crashes in stats and compaction filter for db_ttl_impl
7ea769487 Fix flakiness in column_family_test
a2c4cc756 Don't treat warnings as error when building release
9aa011fa3 Optimize GetRange Function
36a740889 Fix UNLIKELY parenthesis
9bdbaab94 Update USERS.md
2ab7065af build: avoid unused-variable warning
88044340c Add Size-GB column to benchmark reports
d2346c2cf Fix hang with large write batches and column families.
b6b100fe0 Remove iter_refresh_interval_us
197f01b7b Bugfix remove deprecated option use which was removed in previous commit 019ecd19329ee895284e9b040df0ffe4c08b35d8
aa094e8ea Fix conversion from nano-seconds to milli-seconds and seconds
dddceefe5 Fix clang build
d4540654e Optimize GetApproximateSizes() to use lesser CPU cycles.
fd96b5540 Making GetOptions() comment better (#597)
7246ad34d Don't compact bottommost level in SuggestCompactRange
7f47ba0e2 Fix possible SIGSEGV in CompactRange (github issue #596)
aba005c44 Merge pull request #585 from fyrz/RocksJava-RemoveDeprecatedStuff
d6f39c5ae Helper function to time Merges
a087f80e9 Add scripts to run leveldb benchmark
1bb4928da Include bunch of more events into EventLogger
3db81d535 Fix memory leak in cache_test introduced in the previous commit
4961a9622 Fix build
93ab1473d Merge pull request #593 from charsyam/feature/type-1
6ede020dc fix typos
3d1af4ae6 Don't preinstall jemalloc in Travis
242f9b4c2 Fix CLANG build issue introduced in previous commit
794ccfde8 Task 6532943: Rocksdb - SetCapacity() can dynamically change cache capacity if feasible
98a44559d Build for CYGWIN
d01bbb53a Fix CompactRange for universal compaction with num_levels > 1
e003d3864 Abstract out SetMaxPossibleForUserKey() and SetMinPossibleForUserKey
fd7a35731 Enable open source users to use jemalloc (github issue #438)
aa14670b2 Add an assertion in CompactionPicker
2dc421df4 Implement DB::PromoteL0 method
9bf40b64d Print max score in level summary
397b6588b options.paranoid_file_checks to read all rows after writing to a file.
283a04296 Set --seed per test
618d07b06 Making PreShutdown tests more reliable.
0a91bca5d test: avoid vuln-inducing use of temporary directory
6e359419f Add rpath for production builds
78dbd087d Improve benchmark scripts
6a5ffee0c Fix gflags Makefile
108a927f0 Merge pull request #589 from coderplay/patch-1
a58fd7427 Update USERS.md
d85d08c7b One last fix to Makefile
2db96dca1 Fix make install when there is no shared lib
7d136994c Get rid of error output
79c1b021a Fix Makefile
742fa9e31 Fix compile with two gflags
79c21ec0c skip ioctl-using tests when not supported
6059bdf86 Add experimental API MarkForCompaction()
acf8a4141 maint: use ASSERT_TRUE, not ASSERT_EQ(true; same for false
b5400f90f Kill dead code
48b0a045d Speed up reduce_levels_test
00c2afcd3 Fix bug in ExpandWhileOverlapping()
019ecd193 [RocksJava] Remove deprecated methods
98ef21d2f Merge pull request #584 from pshareghi/rocksdb-3.10-falloch
5b7131c75 [RocksJava] Removed deprecated skipLogErrorOnRecovery methods.
566f65271 [RocksJava] Removed deprecated ColumnFamilyDescriptor methods
6997aa0b6 Merge pull request #582 from fyrz/RocksJava-Fix-RateLimiter
582c4b0f7 [RocksJava] Fix RateLimiter Tests in 3.10
6cfb2150d Merge pull request #581 from vladb38/patch-3
d71e8f7ad Update USERS.md
debaf85ef Bug of trivial move of dynamic level
12d7d3d28 Fix and Improve DBTest.DynamicLevelCompressionPerLevel2
a1271c6c6 Fix build break introduced by new SyncPoint interface change
fcb206b66 SyncPoint to allow a callback with an argument and use it to get DBTest.DynamicLevelCompressionPerLevel2 more straight-forward
281db8bb6 Temporarily disable test CompactFilesOnLevelCompaction
e8808b912 Added falloc.h in build_detect_platform
1983fadcb assert(sorted) in vector rep
9da874801 Get benchmark.sh loads to run faster
9b983befa Fix flakiness of WalManagerTest
d41a565a4 Don't do O(N^2) operations in debug mode for vector memtable
08be1803e Fix bad performance in debug mode
0a0501c8d Add Xfunc to makefile
e7ad14926 Fix flakiness in FIFOCompaction test (github issue #573)
abb405227 Kill benchharness
894e9f745 Update Patent Grant.
590fadc40 Fix compile warning on CLANG
47b874398 Make Compaction class easier to use
753dd1fdd Fix valgrind issues in memtable_list_test
697380f3d Repairer documentation improvement.
2f66d7f92 Add LinkedIn back to USERS.md
0feeee643 Fix memtable_list_test
7b9581bc3 Fixed xfunc related compile errors in ROCKSDB_LITE
fabc11569 MemTableList tests
9741dec0e Fix a compile error in ROCKSDB_LITE in db/db_impl.cc
465b25ca9 "make commit-prereq" doesn't really build ROCKSDB_LITE
d2a056241 Fix a compilation error in ROCKSDB_LITE in db/internal_stats.h
316ec80bf fault_injection_test: add a test case to cover log syncing after a log roll
ed229a0de Fixes for readcache-flashcache
91df4e969 Remove use of whole-archive to include jemalloc
84c5bd7eb Add thread-safety documentation to MemTable and related classes
ee9bdd38a Script to check whether RocksDB can read DB generated by previous releases and vice versa
2b019a151 Enabling checksum in repair db as it should have been.
b1bbdd791 Create EnvOptions using sanitized DB Options
edbb08b5d Fix Makefile
199313dc3 build: create .o files specifically for java-related targets
b118238a5 Trivial move to cover multiple input levels
e7adfe690 Fix formatting of USERS.md
4e7543dcf Add USERS.md
58346b9e2 Log writer record format doc.
db6569cd4 Fix the compilation error in flashcache.cc on Mac
cba592001 build: don't use a glob for java/rocksjni/*
c66483c13 Fix github issue #563
de22c7bd1 Integrate Jenkins with Phabricator
f12614070 Fix TSAN build error of D36447
824e64634 Adding another NewFlashcacheAwareEnv function to support pre-opened fd
5e067a7b1 Clean up compression logging
e3ee98b38 run 'make check's rules (and even subtests) in parallel
a45e7581b Avoid naming conflict of EntryType
3be82bc89 Add p99.9 and p99.99 response time to benchmark report, add new summary report
953a885eb A new call back to TablePropertiesCollector to allow users know the entry is add, delete or merge
d2a92c13b avoid returning a number-of-active-keys estimate of nearly 2^64
a7ac6cef1 Fix level size overflow for options_.level_compaction_dynamic_level_bytes=true
089509b84 db_test: clean up sync points in test cleaning up
afbafeaea Disallow trivial move if compression level is different
d0695f3e2 Fix crash caused by opening an empty DB in readonly mode
51c8133a7 Fix make unity build compiler warning about "stats" shadowing global variable
df71c6b9e Script to trigger jenkins test
38a01ed1b Update COMMIT.md
76d63b452 Fix one non-determinism of DBTest.DynamicCompactionOptions
b23bbaa82 Universal Compactions with Small Files
2511b7d94 Makefile minor cleanup
1bd70fb54 Add --stats_interval_seconds to db_bench
fd3dbef22 Clean up old log files in background threads
99ec2412e Make the benchmark scripts configurable and add tests
2158e0f83 Fix clang build
d61cb0b9d db_bench can now disable flashcache for background threads
1c47c433b build: always attempt to update util/build_version.cc
e018892bb Formalize the DB properties string definitions.
cfa576402 Make auto_sanity_test always use the db_sanity_test.cc of the newer commit.
e9fddb7a2 Merge pull request #560 from xiaoxichen/patch-1
bcd8a71a2 Fix interger overflow on i386 arch
030859eb5 Dump compression info on startup
3539e0644 Merge pull request #558 from aamihailov/master
a3e4b3248 fix compilation error (same as fix#284)
ff1ff7c62 TSAN: avoid new link failure with -pg
39d508e34 Add a missing section title in HISTORY.md
727684bf9 Fixed a typo in RocksDBSample.java
ca25e86ef build: cause the "check" rule to depend on $(PROGRAMS)
986bdc680 Merge branch 'master' of github.com:facebook/rocksdb
1e57f2bf2 Fix build
2d417e52d Update HISTORY.md for 3.10.0
cbd6a2073 Merge branch 'master' of github.com:facebook/rocksdb
211ca26ae Fixing build issue
2495f9396 Merge pull request #555 from pshareghi/staticLZ4
f06de5f23 Merge pull request #556 from fyrz/RocksJava-Maven-Fix
cd987c383 Fix compile error when NROCKSDB_THREAD_STATUS is not used.
8f104ae5e [RocksJava] Maven build fix
3d1a924ff Adding stats for the merge and filter operation
afc51649e Merge pull request #546 from fyrz/RocksJava-MemEnv
4806cc126 Added static lz4 support for roccksjavastatic
6284eef4c Merge pull request #545 from fyrz/RocksJava-Level-Compression
248c063ba Report elapsed time in micros in ThreadStatus instead of start time.
315abac94 Undeprecate GetLiveFiles()
a057bb2a8 Improve ThreadStatusSingleCompaction
868968c8e Merge branch 'master' of github.com:facebook/rocksdb
689391406 Make SSTDumpTest.GetProperties less noisy
8d8656243 Merge pull request #551 from fyrz/RocksJava-JavaDoc-Fix
f8c505b23 Merge pull request #550 from fyrz/RocksJava-Sample-Fix
46443bfa9 [RocksJava] Add missing JavaDoc annotations
864b7e88f [RocksJava] Java sample bugfix
2b2394cbb [RocksJava] DBBenchmark option for RocksMemEnv
fd8804f97 [RocksJava] Expose MemEnv in RocksJava Summary: In 3.10 the C++ code was extended with a MemEnv implementation. This is now also available in RocksJava.
004b89fba [RocksJava] Add compression per level to API
51da3aab4 Merge pull request #536 from fyrz/RocksJava-32-Bit-adjustment
b7e88cfb5 Merge pull request #543 from fyrz/RocksJava-Logger-Comment
38d286f14 Clean-up WAL directory before running db_test
28bc6de98 rocksdb: print status error message when (ASSERT|EXPECT)_OK fails
9405b5ef8 rocksdb: Remove #include "util/string_util.h" from util/testharness.h
220d0dff7 rocksdb: Remove #include "util/random.h" from util/testharness.h
b088c83e6 Don't delete files when column family is dropped
17ae3fcbc rocksdb: initial util/testharness clean up
39f4271be [RocksJava] Enhanced Logger comment
5615e23d8 [RocksJava] 32-Bit adjustments
836bcc2f0 Merge pull request #532 from fyrz/RocksJava-LevelCompactionDynamicLevelBytes
67d995808 rocksdb: fix make unity
52e0f3353 Clean up compactions_in_progress_
6b626ff24 rocksdb: change db_test::MultiThreadedDBTest as value parameterized test.
9720ea4de A build option to run through all check-in requirements.
0831a3599 Add a DB Property For Number of Deletions in Memtables
f7ed65464 Fix RocksJava test failure due to deprecation of table_cache_remove_scan_count_limit
51301b869 Enable dynamic changing of rate limiter's bytes_per_second
652db51a3 Fix compilation error in rocksjni/write_batch_test.cc
dfccc7b4e Add readwhilemerging benchmark
c345d1ee8 [RocksJava] Integrated changes for D34857
12350115d [RocksJava] Added LevelCompactionDynamicLevelBytes to Options
230e68727 Fix TSAN failue in env_test
155d468c5 Using chrono as a fallback
81345b90f Create an abstract interface for write batches
46214df4a Only run db_test in Travis
c88ff4ca7 Deprecate removeScanCountLimit in NewLRUCache
b4b69e4f7 rocksdb: switch to gtest
413e35273 Merge pull request #540 from dalgaaf/wip-da-fix-elif
969aa806b util/xfunc.h: fix #elif check for NDEBUG
87c7d49d6 util/env_posix.cc: fix #elif check for __MACH__
c86e5d7b9 stack_trace.cc: fix #elif check for OS_MACOSX
98c37fda5 Remove unused parameter in CancelAllBackgroundWork
9fd6edf81 rocksdb: Replace ASSERT* with EXPECT* in functions that does not return void value
d4d42c02e Fixed clang build in env.h
b2b308652 Speed up rocksDB close call.
a7aba2ef6 rocksdb: Add gtest
95344346a rocksdb: Small refactoring before migrating to gtest
bd4963e64 rocksdb: update reference to the latest version of clang dev in fbcode
56337faf3 Fix compaction IO stats to handle large file counts
eafa1bfc3 Merge pull request #529 from fyrz/RocksJava-Logger
ac03c4d51 Merge pull request #522 from fyrz/RocksJava-Optimize-Filters-For-Hits
c6967a1a5 Make RecordIn/RecordOut human readable
c8da67032 Stop printing per-level stall times.
04778a94c [RocksJava] OptimizeFiltersForHits
57f2a00c6 RocksJava - JNI Logger callback
814627af3 [RocksJava] Change log level at runtime in custom logger
a3bd4142f [RocksJava] Logging JNI callback
58878f1c6 Switch to use_existing_db=1 for updaterandom and mergerandom
12134139e Fixed the unit-test issue in PreShutdownCompactionMiddle
fd1b3f385 Fix the issue in PreShutdownMultipleCompaction
56c4a9c76 Fix compile warning in thread_status_util.h on Mac
417367c42 Fix SIGSEGV when not using cache
e25ff039c Prevent slowdowns and stalls in PreShutdown tests
f69071265 Speed up db_bench shutdown
c1b3cde18 Improve the robustness of ThreadStatusSingleCompaction
8c12426c9 Fix the deadlock issue in ThreadStatusSingleCompaction.
b16ead531 DBTest.DynamicLevelCompressionPerLevel should not run without snappy support
a5e60bafc Fix a typo / test failure in ThreadStatusSingleCompaction
cb2c91850 Don't run some tests is snappy is not present
c594b0e89 Allow GetThreadList() to report operation stage.
2623b2cf6 Include chrono
52d8347a9 EventLogger
756532daf Merge pull request #524 from fyrz/RocksJava-Test-Fix
47a2b3a40 Merge pull request #534 from fyrz/RocksJava-Fix-BrokenJacocoReport
9d22a1f13 Allow negative Wnew
2c84303aa Merge pull request #528 from fyrz/RocksJava-NativeLibraryLoader
2dc636f62 [RocksJava] Fix broken jacoco report
f210b0f6c [RocksJava] Fix NativeLibraryLoader
3ebebfccd Prevent xxhash symbols from polluting global namespace
53996149d Removing unnecessary kInlineSize
b411d0603 Prevent stalls in preshutdown tests
1d43bc41f Fixing segmentation fault in db_bench
e9de8b65a Change the way options.compression_per_level is used when options.level_compaction_dynamic_level_bytes=true
2b785d76b Fixed a bug where CompactFiles won't delete obsolete files until flush.
6f5579868 Fixed a compile error in db_bench in mac.
05d92efa7 Add convenience.cc to src.mk
2884b100b db_bench: Better way to randomize repeated read keys in -read_random_exp_range
284be570c Provide a mechanism to inform Rocksdb that it is shutting down
2ddf53b2c Get OptimizeFilterForHits work on Mac
480b28476 Fix make_new_version.sh
89597bb66 Allow GetThreadList() to report the start time of the current operation.
37921b499 db_bench: Add Option -read_random_exp_range to allow read skewness.
485ac0dbd Add rate_limiter to string options
e126e0da5 Single threaded tests -> sync=0 Multi threaded tests -> sync=1 by default unless DB_BENCH_NO_SYNC is defined
dc4532c49 Add --thread_status_per_interval to db_bench
34c75e984 fix-up patch: avoid new link error
ebc647de8 build: fix missing dependency problems
492f6d27e Fix a segfault in fbson under Mac OS X compiler
0d13bbe27 RocksJava] Fix ColumnFamily tests
67533809f [RocksJava] Fixed CompactionTest
1b7b997b8 [RocksJava] Remove MaxValue from Statistics
f862b3812 [RocksJava] Fix cleanup in tests
a01b59259 [RocksJava] DefaultColumnFamily Memory Fix
22c73d15b [RocksJava] Fix ColumnFamily tests
694988b62 Fix a bug in stall time counter. Improve its output format.
b8d23cdcb Revert chrono use
db0373934 options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically.
f29b33c73 Add functionality to pre-fetch blocks specified by a key range to BlockBasedTable implementation.
c4bd03a97 Fix typo in log message
3cf7f353d Instrument memtable seeks
216a9e16f Fix compile
b9ff6b050 Fix a bug in ReadOnlyBackupEngine
afa8156af adding stdlib to fbson
f9c14a42e Fix compile on Mac
a9f0e2b5b Fix compile
e7c434c36 Add columnfamily option optimize_filters_for_hits to optimize for key hits only
ba9d1737a RocksDB on FreeBSD support
4ba119df5 rocksdb: Update path to dev clang in fbcode
8984e5f84 Fix race in sync point.
03b432d4b rocksdb: Fix uninitialized use error
ccef2a766 Merge pull request #518 from fyrz/RocksJava-Native-Library-Loader-Fix
9fcf1a7b0 [RocksJava] RocksJava Testcases
03bbf718c Return fbson
62247ffa3 rocksdb: Add missing override
1e06a4068 Support builds for MongoDB+RocksDB
b74ad6632 Merge pull request #508 from fyrz/RocksJava-Final-Patch
d9f4875e5 Disable pre-fetching of index and filter blocks for sst_dump_tool.
182b4ceac Limit key range to number of keys, not number of writes
a360bb61b Merge pull request #516 from fyrz/RocksJava-Update-Statistics-To-3.10
8c7684474 [RocksJava] Updated TickerTypes and Histogram to 3.10
7f0c77cb3 [RocksJava] Integrated changes from D33417
819e787bb [RocksJava] Final usage correction
5139e678b Upgrade compiler in Travis
4ade89962 Fix compile error on MacOS.
ace3d8506 Revert "Unused managed iterator"
7b8f348e5 Attempt at fixing travis issue
d85993998 Merge pull request #506 from fyrz/RocksJava-Raw-Use
30e93c9b9 Merge pull request #505 from fyrz/RocksJava-Redundant-Modifier
217854dc4 Introduce DISABLE_WARNING_AS_ERROR in Makefile
bd339a979 Unused managed iterator
174a79c99 LevelDb include guard replaced with #pragma once
6fdda8ac4 rocksdb: changes to support 'make analyze' in Jenkins
96d989f70 catch config errors with L0 file count triggers
62f7a1be4 rocksdb: Fixed 'Dead assignment' and 'Dead initialization' scan-build warnings
5636c4a80 Verbose build in travis
a047409ae Fixed a bug in the test case
4f514a53d build: enable more compiler warnings
a2b911b63 inputs: restore "const" attribute removed by D33759
1b4082581 mark as unused some variables with cpp-derived names
c6d54b503 fix erroneous assert: cast kBlockSize (of type unsigned int) to "int"
aa5d8e6d9 table_test.cc: add missing 5th arg in TestArgs initializer
c37937a9c maint: remove extraneous "const" attribute from return type
9283c7afd build: remove always-true assertions
06a766de5 Adding Flush to AutoRollLogger
92416fa7f Fix mac build
96ab15d30 GetOptionsFromString + fixes to block_based_table_options
73711f956 rocksdb: Fix scan-build bug 'Memory leak' in db/db_bench.cc
98870c7b9 rocksdb: Fix scan-build memory warning in table/block_based_table_reader.cc
a42324e37 build: do not relink every single binary just for a timestamp
d45a6a400 Add rocksdb.num-live-versions: number of live versions
11581b741 build: abbreviate AR command, too
b8ac71ba1 Revert "Fbson to Json"
7ce1b2c19 Fbson to Json
7d817268b Managed iterator
b4b8c25a5 build: factor out AM_LINK command (trivial)
dc885c6e9 build: make "make" output readable by default
a37b46ae1 build: fix Makefile inconsistencies (trivial)
55277c328 build: remove unused rules: rocksdb_shell, DBClientProxy_test
3ad6b794c rocksdb: Fix 'Division by zero' scan-build warning
12753130e Remove ThreadStatusMultiCompaction test
f0c36da6e Add thread_status_util_debug.cc back
daebb1f91 build: running "make" with no arguments must not "uninstall"
e60bc99fe Allow GetThreadList to reflect flush activity.
b9a0213cd build: fix unportable Makefile syntax
4e4b85784 rocksdb: Fix scan-build 'Called C++ object pointer is null' and 'Dereference of null pointer' bugs
b3fd16226 build: remove unportable use of sed in favor of $(CXX)'s -MT
e7ea51a8e Introduce job_id for flush and compaction
6a0e737d9 [RocksJava] Raw use of parametrized class
439701270 [RocksJava] Redundant access-modifier in interfaces
2d62e8051 Merge pull request #504 from fyrz/RocksJava-Flush-Correction
eaf39568e [RocksJava] FlushOptions Correction
5d1151deb Added simple monitoring script to monitor overusage of memory in db_bench
5f00af457 DBTest.DestroyDBMetaDatabase: create DB directories if not exists
68af7811e Remember whole key/prefix filtering on/off in SST file
fd5970b45 Merge pull request #503 from weiweisd/master
513ad866b modify double type euqal compare in json_document.cc
933973dc2 Merge pull request #495 from fyrz/RocksJava-CF-Name-Byte-Handling
5e8e453d5 [RocksJava] Integrated changes from D33165
677d02427 [RocksJava] CF Name shall handle bytes correctly
d1cafc089 Merge pull request #1 from facebook/master
6d6305dd7 Perf Context to report DB mutex waiting time
863009b5a Fix deleting obsolete files #2
1851f977c Added RocksDB stats GET_HIT_L0 and GET_HIT_L1
91ac3b206 Print DB pointer when opening a DB
bee4e5124 Merge pull request #492 from fyrz/logger-logv-virtual
cfe8837e4 Switch logv with loglevel to virtual
aaceef363 Fix formatting
ee4aa9a0e Merge pull request #481 from mkevac/backupable
82faa377a added simple example for db restore from backup
d090330c8 fixed c_simple_example and added some comments
965130830 renamed backup to backup_and_restore in c_test for clarity
bbb52b21f Merge pull request #483 from adamretter/restructure-java-build
7e50ed8c2 Added some more wrappers and wrote a test for backup in C
218c3ecea Fix std::cout data race
8f679c290 Merge branch 'master' of github.com:facebook/rocksdb
da9cbce73 Add Header to logging to capture application level information
2a979822b Fix deleting obsolete files
8e83a9d31 Add a missing field for STATE_MUTEX_WAIT to global_state_table
6f1013035 Fix DestroyDB
7de4e99a8 Revert "Fix wal_dir not getting cleaned"
9a52e06a0 Add GetID to ColumnFamilyHandle
181191a1e Add a counter for collecting the wait time on db mutex.
f36d394ae Fix wal_dir not getting cleaned
53ae09c39 db_test: fix a data race in SpecialEnv
fe9f69119 Fix fault_injestion_test
b37f5ffc7 Put db_test back to TESTS in Makefile
108470e96 Fix stack trace on mac
3e53760fc Fix compaction_picker_test
e39f4f6cf Fix data race #3
e63140d52 Get() to use prefix bloom filter when filter is not block based
678503ebc Add utility functions for interpreting ThreadStatus
4d98e2935 rocksdb: Enable scan-build static analysis
756e1f151 Remove unused util/thread_event_info.h
dad98dd4a Changes for supporting cross functional tests for inplace_update
9898f6398 Divide test DBIteratorTest.DBIterator to smaller tests
829363b44 Options::PrepareForBulkLoad() to increase parallelism of flushes
b04408c47 Fix unity build
8d3819369 NewIteratorWithBase() for default column family
2c2d5ab7e Fix compile warning in util/xfunc.h
0b8dec717 Cross functional test infrastructure for RocksDB.
868bfa403 Merge pull request #488 from ekg/master
9900f3821 Merge pull request #484 from fyrz/RocksJava-Release-MD-change
e6eaf938c remove old debugging message (#487)
f33f3955e Moved Java Samples main classes into samples/src/main/java
ad325517f Update test lib versions and maven plugin versions
d6187d07b Maven can now build a standard project layout
157768890 Moved Java Benchmark main classes into benchmark/src/main/java
dd8d5471e Adjustment to NativeLibraryLoader to allow native library to be loaded from either java.library.path or from extracting from the Jar. Means that the test in the build do not need to rely on the Jar, useful when creating similar builds (and executing tests) from Maven
353db6dae Moved Java main classes into src/main/java
98cb501bc Moved Java test classes into src/test/java
7479a62a7 Release.md - Remove version change instrcution
4a4e4279f Update HISTORY-JAVA.md
384cb6619 Merge pull request #480 from fyrz/RocksJava-Deprecate-SkipLogError
ca52a67cf [RocksJava] Deprecate setSkipLogErrorOnRecovery
114d21878 Merge pull request #479 from fyrz/RocksJava-Snapshot-Sequence-Number
cb5c3159f [RocksJava] Snapshot - GetSequenceNumber
ea189b320 Merge pull request #474 from fyrz/RocksJava-GetUpdatesSince
391f85fc8 [RocksJava] Incorporated changes for D32151
68cd93b87 [RocksJava] GetUpdatesSince support
caedd40dd [RocksJava] Adjusted auto pointer
b39006e3d [RocksJava] enable/disable File deletions
9a456fba2 [RocksJava] GetUpdatesSince support
d3a736761 Merge pull request #482 from fyrz/RocksJava-TTL-Fix
939bb3659 [RocksJava] Fix ColumnFamily name alloc in TTL DB
86e2a1eee Allow creating backups from C
db9ed5fdb Unaddressed comment in previous diff. Change only in code comments.
5917de0ba CappedFixTransform: return fixed length prefix, or full key if key is shorter than the fixed length
6c6037f60 Expose Snapshot's SequenceNumber
2fd8f750a Compile MemEnv with standard RocksDB library
173c52a97 Fix build on older compilers -- emplace() is not available
d07fec3bd make DBTest.SharedWriteBuffer to pass MockEnv
4bdf38b16 Disable FlushSchedule when running TSAN
e84299c76 Fix bug recently introduced in MemFile::Lock()
e5aab4c2b Fix data race in HashLinkList
2113ecd3c Merge pull request #472 from fyrz/RocksJava-Cleanup
10af17f3d fault_injection_test: add a unit test to allow parallel compactions and multiple levels
0c4d1053d Fix data race #5
cc0d8be01 [RocksJava] Integrated review comments (D32145)
5257c9c42 Merge pull request #452 from robertabcd/backupable-mem
560ed402b [minor] fprintf to stderr instead of stdout in test
551a41df3 Merge pull request #476 from alabid/alabid/add-to-simple-example
d2a2b058f fault_injection_test: to support file closed after being deleted
f8f040ccc Updated .gitignore to ignore *~ files and example object files
e8bf2310a Remove blob store from the codebase
ea7d0b943 Added WriteBatch block to simple_example.cc
d6c7300cc Fixed a compile warning in clang in db/listener_test.cc
f9758e012 Add compaction listener.
e919ecedf SuperVersion::Unref() to use sequential consistency to decrease ref counting
4c49fedaf Use ustricter consistency in thread local operations
1b43ab58d fault_injection_test: add more logging and makes synchronization slightly stronger
ca2b00277 [RocksJava] Cleanup portal.h & tests
f8dc5c459 [RocksJava] Add missing test to Makefile
b3c133148 [RocksJava] Removed todo comment in portal.h
7ffcc457f [RocksJava] Cleanup portal.h
c4fb83441 Update the comment for the removal of mac-install-gflags.sh
be8f0b12e Rename DBImpl::log_dir_unsynced_ to log_dir_synced_
c1de6c42a fault_injection_test: add a test case to drop random number of unsynced data
d888c9574 Sync WAL Directory and DB Path if different from DB directory
58f34edfc Fix valgrind
f1c886247 Fix data race #1
b08b2fe73 Merge pull request #471 from fyrz/RocksJava-Fix-NativeLibraryLoader
e61f38e5a [RocksJava] Fix native library loader
26b50783d Fix assert in histogramData
42189612c Fix data race #2
f5a839835 Fix archive WAL race conditions
43ec4e68b fault_injection_test: bring back 3 iteration runs
c2e8e8c1c Fix two namings in fault_injection_test.cc
b4c13a868 fault_injection_test: improvements and add new tests
a52dd0024 Fix ASAN failure with backupable DB
910186c27 Return the build with 4.8.1
401d4205e Add thread sanitizer
b068f0a67 Upgrade our compilers
a76d92862 Merge pull request #466 from fyrz/RocksJava-Support-ReadOptions-Iterator
bef7821f0 [RocksJava] ReadOptions support in Iterators
3b494a610 Make options_test runnable on ROCKSDB_LITE
912c52e82 Merge pull request #465 from fyrz/RocksJava-BlockBasedTable-FormatVersion
cd4c07197 Update HISTORY.md for GetThreadStatus() support on compaction.
46a7048dc Reduce false alarm in ThreadStatusMultipleCompaction test
aed028698 Merge pull request #462 from fyrz/RocksJava-JNI-allocation-correction
e5df90f5d Fix comment (minor)
dd53428f8 Incorporated review comments
908258a4f [RocksJava] BlockBasedTableConfig 3.10
2efe22849 [RocksJava] Incorporated changes for D31809
4e48753b7 Sync manifest file when initializing it
e204a5a16 [RocksJava] ColumnFamily name JNI correction
96264784d [RocksJava] ColumnFamily name JNI correction
ae82849bc Fix build failure
423dee841 Abort db_bench if Get() returns error
206237d12 DBImpl::CheckConsistency() shouldn't create path name with double "/"
5e98e5324 Merge pull request #458 from fyrz/RocksJava-TTLDB-Support
4ffe0be41 [RocksJava] Integrated changes for D31449
e82856754 [RocksJava] Integrated changes from D31449
859c54a03 [RocksJava] TTL-Support
5ff8aec4d [RocksJava] TTL Support
ca47da9e6 [RocksJava] TTL-Support
1190ebe5a Merge pull request #461 from fyrz/RocksJava-DirectSlice-Fix
ea25ff715 [RocksJava] Integrated proposed simplificiation
d68e83c35 [RocksJava] DirectSlice String termination fix
0ddf5f73e memenv: normalize file path
4d9d5955a Merge pull request #464 from fyrz/RocksJava-Various-Fixes
ceaea2b72 Adding prefix_extractor string config parameter
3d628f8f2 Update format_version comment
155bec4cf fallocate also tests FALLOC_FL_KEEP_SIZE
c75c02e7a [RocksJava] WriteBatchWithIndexTest fix
c787fb50b [RocksJava] JavaDoc errors in Java8
b229f970d Remove Compaction::ReleaseInputs().
f2ddb8b45 Fix for bug where GeoDB accesses key after next modification of iterator
d10f1de2b Ported LevelDB's fault_injection_test
2bb059007 Change db_stress to work with format_version == 2
9ab5adfc5 New BlockBasedTable version -- better compressed block format
2355931c6 Merge pull request #450 from adamretter/writebatch-with-index
3d246c89c Abstract duplicate code on key and value slice objects into generic methods
2d0dd8db3 Implement WBWIRocksIterator for WriteBatchWithIndex in the Java API
de678b288 Abstractions for common iterator behaviour
e01acb3a0 Test for WriteBatchWithIndex#newIterator()
56f24941a Simplify the Java API by permitting WriteBatchWithIndex to be provided straight to RocksDB#write
95d5f9848 Test for RocksDB#write(WriteBatchWithIndex)
ef5b34dee Implement WriteBatchWithIndex in the Java API
c6e554561 Abstractions for common write batch behaviour
be905491b Test for WriteBatchWithIndex#newIteratorWithBase(org.rocksdb.RocksIterator)
2241e3f4d Extract the interface for a RocksIterator
a8cfa7ace Extract the interface for a WriteBatch
45e43b81d Adds support for db->DefaultColumnFamily() to the Java API
516a04267 Add LZ4 compression to sanity test
2ccc54301 Merge pull request #460 from neutronsharc/master
2a7bd0ea4 Remove duplicated method declarations in C header.
bb128bfec More accurate message for compaction applied to a different version
96b8240bc Support footer versions bigger than 1
53f615df6 Fix clang build
02b30202c Merge pull request #455 from Andersbakken/stdlib_fix
2159484dd Remove two unnecessary blank lines in db/db_test.cc
e7dd88c57 Merge pull request #441 from fyrz/RocksJava-ColumnFamilyDescriptor-Alignment
d2c018fd5 Make ThreadStatusMultipleCompaction more robust.
bf9aa4dfc Improve GetThreadStatus to avoid false alarm in some case.
c91cdd59c Allow GetThreadList() to indicate a thread is doing Compaction.
402c1152a Fix c_simple_example
a9ea65d65 Build with clang 3.5 on Linux.
23ad5f401 [RocksJava] Incorporated changes for D30525
0aab1005f [RocksJava] ColumnFamilyDescriptor alignment with listColumnFamilies
15d2abbec Fix build issues
abb9b95ff Move compression functions from port/ to util/
9132e52ea DB Stats Dump to print total stall time
93b35c299 Merge pull request #453 from fyrz/SimpleCExampleSigSegv
628a67b00 Reduce memory footprint in backupable db.
ef3901642 Fixed memory issue in c_simple_example
b89d58dfa :%s/build_config/make_config
242b9769c Memtablerep Benchmark
73ee4feba Add comments about properties supported by DB::GetProperty() and DB::GetIntProperty()
2dca48f55 Merge pull request #451 from StanislavGlebik/document_db_improvement
4b57d9a82 Fixed negative numbers comparison in DocumentDB
9ef59a09a VersionSet::AddLiveFiles() to assert current version is included.
4d16a9a63 VersionBuilder to optimize for applying a later edit deleting files added by previous edits
7731d51c8 Simplify column family concurrency
07aa4e0e3 Fix compaction summary log for trivial move
9d5bd411b benchmark.sh won't run through all tests properly if one specifies wal_dir to be different than db directory.
62ad0a9b1 Deprecating skip_log_error_on_recovery
fa0b126c0 Fix corruption_test -- if status is not OK, return status -- during recovery
d7b4bb62a Fail DB::Open() on WAL corruption
9619081d9 Merge pull request #449 from robertabcd/improve-backupable
49376bfe8 Fix errors when using -Wshorten-64-to-32.
a8c5564a9 Do not issue extra GetFileSize() calls when loading BackupMeta.
caa1fd0e0 Improve performance when loading BackupMeta.
e9ca35815 Fix CLANG build for db_bench
bf287b76e Add structures for exposing thread events and operations.
a801c1fb0 db_bench --num_hot_column_families to be default off
2067058a6 Dump routine to BlockBasedTableReader (valgrind)
ddc81440d db_bench to add an option as number of hot column families to add to
a944afd35 Fixed a compile error in db/db_impl.cc on ROCKSDB_LITE
7ea7bdf04 Dump routine to BlockBasedTableReader
ae508df90 Clean up compile for c_simple_example
b62300961 Fix compile of compact_file_example
ded26605f Merge pull request #444 from adamretter/java-api-fix
98490bccf Fix the build on Mac OS X
4d9972974 Merge pull request #443 from behanna/master
5045c4394 add support for nested BlockBasedTableOptions in config string
d232cb156 Fix the build with -DNDEBUG.
45bab305f Move GetThreadList() feature under Env.
4fd26f287 Only execute flush from compaction if max_background_flushes = 0
0acc73881 Speed up FindObsoleteFiles()
d8c4ce6b5 Merge pull request #442 from alabid/alabid/fix-example-typo
949bd71fd fix really trivial typo
f8999fcf3 Fix a SIGSEGV in BackgroundFlush
ade4034a9 MultiGet for DBWithTTL
fdb6be4e2 Rewritten system for scheduling background work
a3001b1d3 Remove -mtune=native because it's redundant
e27c84522 Merge pull request #437 from fyrz/RocksJava-SliceTests-Fixes
1fed1282a [RocksJava] Incorporated changes D30081
5b9ceef01 [RocksJava] JavaDoc correction
5fbba60b6 [RocksJava] Incorporated changes D30081
b0230d7e0 [RocksJava] Incorporate additions for D30081
b015ed0ca [RocksJava] Slice / DirectSlice improvements
4d422db01 Merge pull request #430 from adamretter/increase-parallelism
04c4e4969 Merge pull request #411 from fyrz/RocksJava-RangeCompaction
62d19b7b5 Merge pull request #427 from haneefmubarak/c-examples
28424d734 style fixes in c example
7198ed5a2 Handle errors during pthread calls
91c58752f error detection and memory leaks in c example
25f70a5ab Avoid unnecessary unlock and lock mutex when notifying events.
7661e5a76 Move the file copy out of the mutex.
17e84f215 Rudimentary test cases for setIncreaseParallelism
eda0dcdd9 Exposed IncreasedParallelism option to Java API as setIncreasedParallelism
efc94ceb2 [RocksJava] Incorporated changes for D29283
69188ff44 [RocksJava] CompactRange support
48adce77c [RocksJava] CompactRange support
153f4f071 RocksDB: Allow Level-Style Compaction to Place Files in Different Paths
06eed650a Optimize default compile to compilation platform by default
cef6f8439 Added 'dump_live_files' command to ldb tool.
7ab1526c0 Add an assert and avoid std::sort(autovector) to investigate an ASAN issue
74b3fb6d9 Fix Mac compile errors on util/cache_test.cc
d7a486668 Improve scalability of DB::GetSnapshot()
ee95cae9a Modifed the LRU cache eviction code so that it doesn't evict blocks which have exteranl references
0ab0242f3 VersionBuilder to use unordered set and map to store added and deleted files
e93f044d9 add range scan test to benchmark script
cb82d7b08 Fix#434
046ba7d47 Fix calculation of max_total_wal_size in db_options_.max_total_wal_size == 0 case
1b7fbb9e8 Update HISTORY.md for release 3.9
635c61fd3 Fix problem with create_if_missing option when wal_dir is used
2871bc7bc Merge pull request #422 from fyrz/RocksJava-Quality-Improvements
8c5781666 Add -fno-exceptions flag to ROCKSDB_LITE.
1f04066ca Add DBProperty to return number of snapshots and time for oldest snapshot
6436ba6b0 Provide mechanism to restart tests from previous error
d84b2bade Replace exception by abort() in dummy HdfsEnv implementation.
9260e1ad7 Bump version to 3.9
8f4e1c1c9 Remove the compability check on log2 OS_ANDROID as it's already blocked by ROCKSDB_LITE
c4a7423c1 Replace runtime_error exception by abort() in thread_local
a94d54aa4 Remove the use of exception in WriteBatch::Handler
a5d4fc0a2 Fix compile warning in db_stress
1a8f4821a Replace exception by assertion in autovector
97c194088 Fix compile warning in db_stress.cc on Mac
5f719d720 Replace exception by setting valid_ = false in DBIter::MergeValuesNewToOld()
c0dee851c Improve formatting, add missing newlines
815f638cd Fix java build
32a0a0384 Add Moved(GB) to Compaction IO stats
a14b7873e Enforce write buffer memory limit across column families
3e684aa68 Integrated changes from D29571
37d73d597 Fix linters
a15169f2e Fixed a Lint problem
b7f9e644c [RocksJava] Quality improvements
e002a6122 [RocksJava] Comparator tests for CF
335e6ad5c [RocksJava] Remove obsolete dbFolder cleanup
b036804ac RocksJava - FindBugs issues
9a632b4a9 Merge pull request #429 from fyrz/RocksJava-MacOSX-strip-fix
b42667506 [RocksJava] MacOSX strip support
e463cb0bc Merge pull request #424 from eile/master
91d898163 Tweak Makefile for building on BG/Q
c6f31a289 minor memory leak in C example
703ef66a8 Merge pull request #426 from fyrz/RocksJava-Restore-PrecisionFix
ac4ed1e30 fix examples/makefile for C example
d7f5ccb0c add c example to makefile and fix "make clean"
9c34d5e36 fix type in C simple example
0a9a7e753 added C version of simple_example
bcf908689 Block Universal and FIFO compactions in ROCKSDB_LITE
67cb7ca75 [RocksJava] Fixed MacOS build of RocksJava
b8136a7d2 Merge pull request #398 from fyrz/RocksJava-CreateCheckPoint
533592a27 Merge pull request #401 from fyrz/RocksJava-Sigsegv-MergeOperatorName
73d72ed5c Block ReadOnlyDB in ROCKSDB_LITE
e47f0fa9e Merge pull request #425 from adamretter/macosx-clean-fix
ff0cb90d1 Do not delete Java Fatal Error Log, developers may still want these for reference
2a792cd30 There will also be a librocksdbjni-osx.jnilib.dSYM folder on MacOSX builds to be deleted
beb74c14c Fix travis-build error
a486352e0 Merge pull request #423 from zerebubuth/c_iterate_upper_bound
26109d487 Store upper bound `Slice` with the same lifetime as the `ReadOptions` so that we can provide a pointer to it.
a97314219 Fix compile error in ROCKSDB_LITE
9d5019327 Replace log2 by implementing Log2 in options_builder
805bac6d2 Add test for upper bounds on iterators using C interface.
f193deea3 [RocksJava] Addressed comments in D28971
94f70a86b [RocksJava] Incoroporated changes for D29013
a280af2a5 [RocksJava] Sigsegv fix for MergerOperatorByName
fcc2dfd9f [RocksJava] Support for stored snapshots
274ba6270 Block internal_stats in ROCKSDB_LITE
4f2e8bab5 Merge pull request #421 from fyrz/RocksJava-PrecisionFix
c4765dc10 [RocksJava] Fix precision problem in rocksjni
14788e181 Merge pull request #420 from rdallman/add-wal
88dd8d889 c api: add max wal total to opts
7e608e2fe Block plain_table_index.cc in ROCKSDB_LITE
13de000f0 Add rocksdb::ToString() to address cases where std::to_string is not available.
90ee85f8e Improve listener_test to avoid possible false alarm
2946e37a0 remove unreliable test in db/cuckoo_table_db_test.cc
9c7ca65d2 free builders in VersionSet::DumpManifest
7530c75ab Merge pull request #413 from saghmrossi/master
d699d7034 Make RocksDB compile without gflags
325722149 Fixes valgrind error in GetSnapshotLink. Free checkpoint now.
ada3d7873 Merge pull request #415 from fyrz/RocksJava-Makefile
569853ed1 Fix leak when create_missing_column_families=true on ThreadStatus
c4b65f70f [RocksJava] Makefile correction
141018016 Make arena use hugepage if possible
3a40c427b Fix db_bench on CLANG mode
9222a2d02 Fixed iOS build caused by GetThreadList feature.
aa31fc506 Improve listener_test by ensuring flushes are completed before assert.
7ec71f101 Provide default implementation of LinkFile, don't break the build
cd278584c Clean up StringSplit
d84069995 Fix mac compile
4f882924d Merge pull request #404 from fyrz/RocksJava-Backup-Restore-3.8
4b63fcbff Add enable_thread_tracking to DBOptions
bafce6197 first rdb commit
9e285d423 Added CompatibleOptions for compatibility with LevelDB Options
353307758 Add IOS_CROSS_COMPILE to macro guard for GetThreadList feature.
eecdebe65 Fixed the destruction order of static variables in ThreadStatusImpl.
004f416b7 Moved checkpoint to utilities
beabc6879 Fixed ~ThreadStatusImpl().
faa8d32be [RocksJava] Integrated changes from D29019.
3d78c7a8c [RocksJava] Lint adjustments
d7529b2de [RocksJava] Cleanup Backupable implementations
fa703efb2 [RocksJava] Improved BackupableDBTest
24fdc4741 [RocksJava] Backupable/Restorable DB update 3.8.0
9972f969e [RocksJava] BackupableDBOptions alginment + 3.8
fbc42a093 Fixed -Werror=unused-but-set-variable in thread_status_impl
a564be715 Fix asan error in thread_status_impl.cc
7165d1886 Fix clang compile error
d0c5f28a5 Introduce GetThreadList API
1fd1aecb3 Merge pull request #409 from fyrz/RocksJava-Make-cleanup
91c8dcefc [RocksJava] Strip library in publish
e7fcaa4d9 [RocksJava] JavaDoc is executed too often
2cd1794e4 [RocksJava] Make cleanup - Clean Target
be005e17b fix clang compilation
5e69f19c4 Merge pull request #405 from fyrz/RocksJava-Convenient-Options
55a344872 Merge pull request #408 from fyrz/Missing-include
9e9a83baf Missing header in build on CentOS
91ccc8ebe [RocksJava] Integrated changes in D29025
5249d0db5 [RocksJava] Convenience methods for Options
8d3f8f969 remove all remaining references to cfd->options()
1e4a45aac remove cfd->options() in DBImpl::NotifyOnFlushCompleted
517c28994 Options helper supports k, m, g, and t unit suffixes
c46c2be8d Merge pull request #397 from fyrz/RocksJava-GetIntProperty
8efd4bb42 [RocksJava] Improved comments in RocksDB class
5529c1ad1 [RocksJava] GetIntProperty in RocksDB
db59eeb61 Merge pull request #406 from fyrz/Build-Fix
e97f014b9 [RocksJava] JavaDoc corrections - Java8
98e59f981 Fixed a bug which could hide non-ok status in CompactionJob::Run()
ec24bd4e6 Merge pull request #402 from adamretter/bugfix-native-library-loader
d3c4a0f4a Improve the comment in InfoLogLevelTest.java
a77e97c53 Merge pull request #396 from fyrz/RocksJava-LogLevel
585c759cf Make sure to use the correct Java classloader for loading the RocksDB Native Library
c3915abba Minor tidyup and use Java 7 for file copying
a122a42bb Merge pull request #399 from fyrz/RocksJava-Version-to-3.8
b8d5e3f08 [RocksJava] MVN Build reads version from version.h
23295b74b Clean job_context
0ce38fe98 Fix signed/unsigned compile
e7960c03a Don't parallelize the build in travis
84af2ff8d Clean job context in DeleteFile
8a1bcc39c [RocksJava] Bump version to 3.8 in rocksjni.pom
5c04acda0 Explicitly clean JobContext
4947a0674 [RocksJava] Incorporated review comments D28947
07cd3c42a [RocksJava] LogLevel support in Options
26dc5da96 Fix compaction_job_test
5f583d2a9 Merge pull request #394 from lalinsky/cuckoo-c
353303a76 Merge pull request #380 from fyrz/RocksJava-Junit-Framework
3f9c95a51 [RocksJava] Minor lint correction
e46450da6 [RocksJava] Rebased + integrated CF tests
cd82beb0c [RocksJava] Merged in latest changes.
b6abab8b7 [RocksJava] Merged & rebased to HEAD
74057d6d2 [RocksJava] Improved tests within RocksJava
628e39e97 [RocksJava] Integrated review comments from D28209
a4b28c1ae [RocksJava] Extended Testcases
36f3a0bb8 [RocksJava] Integrated review comments from adamretter in D28209
b09268695 [RocksJava] Extended testcases
9bec23c41 [RocksJava] Test-framework integration
f617135d5 [RocksJava] Testcase improvements
1fe7a4c62 [RocksJava] Test-framework integration
04ca7481d Fix build
6c1b040cc Provide openable snapshots
9be338cf9 CompactionJobTest
c9fd03ec5 Update docs for NewAdaptiveTableFactory
e6c3cc657 Add very basic tests to make sure the C cuckoo table options compile and run
7fe247080 Update HISTORY.md for RocksJava
c44a29278 Add cuckoo table options to the C interface
136b8583b Merge pull request #395 from lalinsky/fix-env-test
94fa542f8 Update HISTROY.md for 3.8 release
a177742a9 Make db_stress built for ROCKSDB_LITE
746cfaac5 Relax the block count check on deallocation in env_test
f822129b3 Add a unit test for behavior when merge operator and compaction filter co-exist.
4161de92a Fix SIGSEGV
373c665ed Fix broken test in 31b02d.
772bc97f1 No CompactFiles in ROCKSDB_LITE
1d1a64f58 Move NeedsCompaction() from VersionStorageInfo to CompactionPicker
cd0980150 Add concurrency to compacting SpatialDB
3c92e5233 Fix include
25f273027 Fix iOS compile with -Wshorten-64-to-32
fa50abb72 Fix bug of reading from empty DB.
31b02dc21 Improve Backup Engine.
1033db29f Merge pull request #390 from fyrz/RocksJava-Cleanup
9a03da773 Merge pull request #375 from fyrz/RocksJava-ColumnFamilyOptions-Extension-3.6
d50c68e3a [RocksJava] JavaDoc cleanup warnings with Java8
079d942ea [RocksJava] Code-cleanup + Java7 warnings removed
9a255b95f [RocksJava] Sample and Default value
9d2ba2136 [RocksJava] Incorporated review comments
fa9cfc65f [RocksJava] Integrated Review comments from yhchiang in D28023
75010d208 [RocksJava] ColumnFamily custom Options API extension
0345c2156 [RocksJava] Extend Options with ColumnFamilyOptions implementation ColumnFamilyOptions implementation with tests [RocksJava] Extended ColumnFamilyTest
975949522 Fixed clang compile error in version_builder_test
581141935 Fixed GetEstimatedActiveKeys
1f621e6ab Fix additional -Wshorten-64-to-32 errros
767777c2b Turn on -Wshorten-64-to-32 and fix all the errors
113796c49 Fix NewFileNumber()
625e162c6 Merge pull request #393 from fyrz/RocksJava-Flush
fc6fcbab9 [RocksJava] Flush functionality
8e5547f64 [RocksJava] Makefile restructured
26f0a78b0 Merge pull request #363 from adamretter/write_batch-iterate
35c8c814e Make ForwardIterator::status() more efficient
d88568c68 Move -Wnon-virtual-dtor to c++ flags
c7ee9c3ab Fix -Wnon-virtual-dtor errors
746252197 Merge pull request #384 from msb-at-yahoo/compaction-filter-2-empty-changed-values
dd726a59e Bump Version Number to 3.8
4a3bd2bad Optimize usage of Status in CompactionJob
bcdb9671c Fix build
d904fbbb0 Addresed comments from code review https://reviews.facebook.net/D27567
eeb9cf6c4 Test for WriteBatchHandler
8fb4751d5 Iterator support for Write Batches
00211f9c5 Fix SIGSEGV in db_stresS
a4a2bfd6b Merge pull request #391 from Liuchang0812/fixmake
856059094 Merge pull request #389 from Liuchang0812/master
01a770637 remove unused target
dc3410463 Merge branch 'fix-example'
e7620536c fix make static_lib error
543df158c Expose sst_dump functionality as library call.
e3d3567b5 Get rid of mutex in CompactionJob's state
344edbb04 Fixed the shadowing in db/compaction.cc and include/rocksdb/db.h
b8b390342 Fixed compile error in db/db_impl.cc
b622ba5d6 Fixed compile error in db/flush_job.cc
642ac9d8a Fixed compile error in db/compaction.cc and db/compaction_picker.cc
68effa034 Fix -Wshadow for tools
844786189 Fixed -WShadow errors in db/db_test.cc and include/rocksdb/metadata.h
28c82ff1b CompactFiles, EventListener and GetDatabaseMetaData
5c9309053 Turn on -Wshadow
31342c400 Fix implicit compare
a0f887c9e Fix compile
53af5d877 Redesign pending_outputs_
ec101cd49 Correctly test both compaction styles in CompactionDeletionTriggerReopen
8d87467bb Make PartialCompactionFailure Test more robust again.
64d302d30 make DropWritesFlush deterministic
cd5c0925a Merge pull request #387 from fyrz/RocksJava-WShadow-Fix
c4bf07c24 [RocksJava] -WShadow improvements
e526b7140 Make PartialCompactionFailure Test more robust.
0c2be0de3 Turn on -Wshadow for travis
5fd33d26f Turn off -Wshadow
9f20395cd Turn -Wshadow back on
c02338a69 update HISOTRY.md for new release
367a3f9cb Improve DBTest.GroupCommitTest: artificially slowdown log writing to trigger group commit
b52b144c7 Merge pull request #386 from EugenePig/java8
ac95ae1b5 Make sure WAL is synced for DB::Write() if write batch is empty
59d549798 suppress JDK8 errors for #385
ea18b944a Add db_bench option --report_file_operations
2ea1219eb Fix RecordIn and RecordDrop stats
e4211d10c Apply InfoLogLevel to the logs in util/env_hdfs.cc
76f6c7c7c CompactionFilterV2: eliminate an often unnecessary allocation.
29a9161f3 Note dynamic options in options.h
fd24ae9d0 SetOptions() to return status and also add it to StackableDB
b1267750f fix the asan check
83bf09144 Bump verison number to 3.7
da5daa061 Replace some ASSERT_TRUE() asserts in DBTest.DynamicMemtableOptions and DynamicCompactionOptions with more specific ones
b0cda4a11 DBTest.DynamicMemtableOptions to use single background compaction
8810850dd Apply InfoLogLevel to the logs in db/compaction_job.cc
71783f652 Merge pull request #377 from fyrz/RocksJava-KeyMayExist
614bbcbe2 Merge pull request #374 from fyrz/RocksJava-DBOptions-Extension-3.6
d8e119663 Apply InfoLogLevel to the logs in db/version_set.cc
2a019f1d0 Apply InfoLogLevel to the logs in db/wal_manager.cc
469d474ba Apply InfoLogLevel to the logs in db/db_impl.cc
ac6afaf9e Enforce naming convention of getters in version_set.h
09899f0b5 DB::Open() to automatically increase thread pool size if it is smaller than max number of parallel compactions or flushes
636e57b52 Fix coverage script
30ca3752b Revamp our build tools
051c67ff5 Merge pull request #378 from baotiao/master
d0e7e49ae Merge pull request #379 from fyrz/RocksJavaBuildFix
94e31ac22 [RocksJava] Extend Options with DBOptions implementation [RocksJava] Included DBOptionsTest and refactored OptionsTest
b060d3006 [RocksJava] Build fix after options refactoring
9fd65e566 add make clean in examples makefile
8e79ce68c Revert "Fix lint errors and coding style of ldb related codes."
45a612f99 Revert "Fix incorrect fixing of lint errors in ldb_cmd.cc"
27129c739 [RocksJava] KeyMayExist w/o ColumnFamilies
86905e3cb Move VersionBuilder logic to a separate .cc file
74eb4fbe9 CompactionJob
8ddddd62d Fix incorrect fixing of lint errors in ldb_cmd.cc
46c14c666 Fix#258. benchmarkharness -- make bm_min_usec uint
72cb7cf20 Add fsync / corrupt simulation to env_mem
0e526eb9d introduce TestMemEnv and use it in db_test
8db24f4b3 exclude mock test file from MOCK_SOURCES
5594d446f unfriend DBImpl and InternalStats from VersionStorageInfo
82e3ae540 fix c_test
bc9f36fd5 Fix lint errors and coding style of ldb related codes.
c645250ee CompactionStats to support larger value of RecordIn and RecordDrop
f7e6c856a Fix BaseReferencedVersionBuilder's destructor order
c76dcb44d fix
b452dede5 fix
29d83cc33 temporarily remove -Wshadow
c1a924b9f Move convenience.h to /include
7e01d1202 Add support for in place update for db_stress
9f7fc3ac4 Turn on -Wshadow
98849a35f Apply InfoLogLevel to the logs in table/block_based_table_reader.cc
4d2ba38b6 Make VersionBuilder unit testable
2b1f23dca Apply InfoLogLevel to the logs in db/db_iter.cc
ccaf1aa7c Merge pull request #372 from fyrz/RocksJava-CF-Merge-Hardening
a29118ffc Merge pull request #355 from fyrz/RocksJava-Options-Refactoring-3.6
85b04ca76 [RocksJava] Review comments - reformatted MergeTest
df7abb4e8 [RocksJava] Integrated code review comments
171be0ed5 Merge with ColumnFamilies & Hardening CFHandle
39464a990 [RocksJava] Options Refactoring 3.6
0f7f3b860 Check InfoLogLevel earlier in Log functions.
73605d917 Apply InfoLogLevel to the logs in util/db_info_dumper.cc
fda592d90 Merge pull request #356 from fyrz/RocksJava-TableOptions-3.6
c73d13bb8 [RocksJava] Integrate review comments from yhchiang
b011e201f Integrated review comments by ankgup87
2c1bd8846 BlockBasedTableConfig & PlainTableConfig enhancements
e770d6175 Merge pull request #371 from dlezama/master
41af0f56b Fix build break because of unsigned/signed mismatch
c5db7f260 Fix CompactionPickerTest.Level1Trigger2
37e9b6370 Apply InfoLogLevel to the logs in utilities/ttl/db_ttl_impl.h
217cc217d Apply InfoLogLevel to the logs in table/meta_blocks.cc
635905481 WalManager
fd95745a5 Fix compile error in table/plain_table_index.cc
c3dd0f75d comparator_db_test to cover more irregular comparators
6afafa369 Apply InfoLogLevel to the logs in utilities/merge_operators/uint64add.cc
e7ad69b9f Apply InfoLogLevel to the logs in table/plain_table_index.cc
bbd9c5345 Apply InfoLogLevel to the logs in table/block_based_table_builder.cc
065766b8d DynamicCompactionOptions: relax the check bound a little
5c82a8837 Add a test in compaction_picker_test to test the max score
86de2007b Add ComparatorDBTest to test non-default comparators
17be187ff dummy var to suppress compiler warning/error
c2999f54b Revert "tmp"
76d1c28e8 Make CompactionPicker more easily tested
01e6f8509 Apply InfoLogLevel to the logs in db/transaction_log_impl.h
082e49ba8 Apply InfoLogLevel to the logs in db/repair.cc
c4b468000 Apply InfoLogLevel to the logs in db/flush_job.cc
34d436b7d Apply InfoLogLevel to the logs in db/column_family.cc
cda9943f9 Apply InfoLogLevel to the logs in db/compaction_picker.cc
7b3a618f9 Apply InfoLogLevel to the logs in db/db_filesnapshot.cc
2d4fe048f remove dead code
9ab013236 tmp
76d54530d minor - remove default value for ChangeFilterOptions() and ChangeCompactionOptions()
44f0ff31c use fallocate(FALLOC_FL_PUNCH_HOLE) to release unused blocks at the end of file
97451f837 add an env var ROCKSDB_TESTS_FROM to control where to start from a list of tests
e130e88bc DBTest: options clean up - part 4
34f3c5a20 DBTest: options clean up - part 3
cdc7230e4 DBTest: options clean up - part 2
5a921b895 DBTest: options clean up - part 1
c9c935923 Move the check to the beginning of the loop in VersionEdit::EncodeTo()
2110e43a5 Remove an unnecessary include file in version_edit.cc
412b7f85b Include atomic in mock_table.h
c08285334 Include all the mocks
abac3d647 TableMock + framework for mock classes
fb3f8ffe5 Improve the robustness of PartialCompactionFailure test again.
60fa7d136 Improve the robustnesss of PartialCompactionFailure test.
3772a3d09 Fix the bug where compaction does not fail when RocksDB can't create a new file.
c49dedbe0 Merge pull request #366 from fyrz/RocksJava-Backup-Restore-Improvements
a39e931e5 FlushProcess
efa2fb33b make LevelFileNumIterator and LevelFileIteratorState anonymous
f7c973069 [RocksJava] Integrated review comments
7e12ae5a2 [RocksJava] - BackupInfos & Restore-/BackupableDB enhancements
eb357af58 unfriend ForwardIterator from VersionSet
f981e0813 unfriend ColumnFamilyData from VersionSet
834c67d77 rename FileLevel to LevelFilesBrief / unfriend CompactedDBImpl
a28b3c438 unfriend UniversalCompactionPicker,LevelCompactionPicker and FIFOCompactionPicker from VersionSet
5187d896b unfriend Compaction and CompactionPicker from VersionSet
75d7e2c37 Merge pull request #369 from fyrz/Small-Fix
db52419cf Merge pull request #290 from vladb38/master
45e756f04 [RocksJava] Minor correction to the previous pull request merge
f94f1a97d Merge pull request #368 from fyrz/RocksJava-Hardening-RocksIterator
b08c39e14 [RocksJava] RocksIterator: Assert for valid RocksDB instance & documentation
b680033e6 Include atomic in env_test
56ef2caaa [RocksJava] - Hardening RocksIterator
c1c68bce4 remove atomic_pointer.h references
7c303f0e7 Include atomic
179c23021 Merge pull request #351 from fyrz/RocksJava_Snapshot_Support
48842ab31 Deprecate AtomicPointer
f37048ad1 Merge pull request #367 from fyrz/FixBrokenJavaBuild
679a9671f RocksJava Fix after MutableCFOptions change.
714c63c58 db_stress for dynamic options
f1841985e dynamic inplace_update options
a04929aa4 fixed conflict in java/Makefile
a1bae76c8 Integrated changes due to review bei ankgup87
b8ce52648 [RocksJava] Support Snapshots
bc3bc4bc2 Merge pull request #357 from fyrz/JavaTest-Fix
42f0eaceb Merge pull request #354 from fyrz/RocksJava-memtables-3.6
965d9d50b Fix timing
001ce64dc Use chrono for timing
240ed0cd7 Fix uninitialized parameter caused by D24513
724fba2b3 Improve the log in Universal Compaction to include more debug information.
720c1c056 fix erro during merge
b794194ad Remove java build from travis
122f98e0b dynamic max_mem_compact_level
1fee591e7 comments for DynamicCompactionOptions test
574028679 dynamic max_sequential_skip_in_iterations
bd4fbaee3 Fixed cross platform build after introducing Java-7 dependencies
1eb545721 Fix incorrectly merged Java - Makefile
9aa9668a8 [RocksJava] Memtables update to 3.6
4b1786e95 Fix SIGSEGV when declaring Arena after ScopedArenaIterator
05b2e60dd Merge pull request #362 from adamretter/travis-jcheck
9383922cc Added java tests to travis build
90f156402 Fix CompactBetweenSnapshots
2717422e2 Merge pull request #361 from adamretter/fix-yosemite-build
3b5fe3a1f Correct the log message in VersionEdit
c584d2b53 Fix for building RocksDB Java on Mac OS X Yosemite
2a8e5203d db_bench: --batch_size used for write benchmarks too
d755e53b8 Printing number of keys in DB Stats
839c376bd fix table_test
0fd985f42 Avoid reloading filter on Get() if cache_index_and_filter_blocks == false
e11a5e776 Improve the comment of util/thread_local.h
6398e6a6a Fix DeleteFile() + enable deleting files oldest files in level 0
0d3145198 Merge pull request #231 from adamretter/master
a6fb7f312 Fix code review comments raised in https://reviews.facebook.net/D22779
c63494fb6 Tests for ComparatorOptions, Comparator and DirectComparator, and by proxy we also exercise Slice and DirectSlice
5e2527411 Fix code style problems identified by lint
25641bfc9 Fix to memory dealocation when creating a slice from a byte buffer
fc12cb83f Add locking to comparator jni callback methods which must be thread-safe
d6fe8dacc Feature - Implement Java API for Comparator and Slice. Allows use of either byte[] or DirectByteBuffer for accessing underlying data.
700f6ec3f Ignore IntelliJ idea project files and ignore java/out folder
5bfb7f5d0 db_bench: seekrandom can specify --seek_nexts to read specific keys after seek.
ff8f74c20 remove checking lower bound of level size
2dd9bfe3a Sanitize block-based table index type and check prefix_extractor
dbcfe27d6 Merge branch 'master' of https://github.com/facebook/rocksdb
6c6691864 Speed up DB::Open() and Version creation by limiting the number of FileMetaData initialization.
5db9e7664 Fix Mac compile error: C++11 forbids default arguments for lambda expressions
f4363fb81 Fix DynamicMemtableOptions test
8f01bf80c Merge pull request #353 from fyrz/FixBloomfilterRocksJava
ee80fb4b4 Total memtables size counter
c12f571d3 Fix mac compile, second try
d2e60f5ce Fix mac compile
274dc81c9 fix build failure
d6c8dba72 Log MutableCFOptions in SetOptions
4d5708aa5 dynamic soft_rate_limit and hard_rate_limit
065a67c4f dynamic disable_auto_compactions
dc50a1a59 make max_write_buffer_number dynamic
6a150c011 ldb: support --fix_prefix_len
bafbc23ba Filters getting disposed by System.gc before EOL
ca250d71a Move logging out of mutex
5cc9adf5b WriteBatchWithIndex's Iterator bug of SeekToFirst() and SeekToLast()
7658bcc1e Merge pull request #352 from fyrz/OptionsTestMergeProblem
1b97934a2 Options correction
2ef3ed86f Integrated feedback from ankgup87
a40ce219b Adding merge functions to RocksDBJava
cc6c883f5 Stop stopping writes on bg_error_
4942f6efb Merge pull request #350 from fyrz/reenable_accidentally_disabled_test
ee28f431d With the last commit a Test was accidentally disabled. This commit solves this.
b5dd7eed6 Merge pull request #319 from fyrz/column_families_java
18004d2f2 [RocksJava] Column family support
5908d08a0 Merge pull request #336 from fyrz/32BitRocksJavaBug
9e5f2a952 Merge pull request #344 from fyrz/java-doc-enhancements
4f5a68725 32-Bit RocksJava resolution for jlong overflows
16d2ebdbc Minor adjustment to prevent two warnings
70294c911 JavaDoc improvements on RocksJava
833357402 WriteBatchWithIndex supports an iterator that merge its change with a base iterator.
4f65fbd19 WriteBatchWithIndex's iterator to support SeekToFirst(), SeekToLast() and Prev()
f441b273a WriteBatchWithIndex to support an option to overwrite rows when operating the same key
3ead857a0 Fixed Mac compile error in util/options_test.cc
5a7618634 Fixed compile error on Mac: default arguments for lambda expressions
cd0d581ff convert Options from string
f18b4a484 minor update to benchmark script
b7d3d6ebc db_bench: set thread pool size according to max_background_flushes
c5f54a8f3 Merge pull request #339 from fyrz/bloomFilterSupportNewFormat
ced612957 Improved JavaDoc
5e43155b3 RocksJava should support not only BlockBased Bloomfilter
1d525891b Update HISTORY for 3.6
88edfd90a SkipListRep::LookaheadIterator
6a443309d Merge pull request #342 from fyrz/java_makefile_fix
4f272408c RocksJava Makefile includes incorrect paths to version.h
f78b832e5 Log RocksDB version
25f6a852e add db_test for changing memtable size
daab6dc51 Merge pull request #318 from criccomini/master
63eade401 Fix error introduced by merge
ba882972f Merge pull request #340 from nbougalis/nullderef
6bcff9dc2 Merge pull request #341 from fyrz/arc-lint
d6169954b Removed code which prevents `arc lint` from working properly.
b87db0715 Avoid dereferencing a null field
1a1b95347 Merge pull request #270 from tdfischer/check-with-unity
e107b6b2a Merge pull request #337 from fyrz/cross-platform-jdb_bench
70e401a2e Merge pull request #338 from fyrz/rm_obs_code_write_batch_with_index_test
1e5a52815 update release readme
d44871e80 fix java doc directory in git ignore
2a4d6e796 merge master to resolve merge conflicts
6b2c1d962 make publish jni jars depend on release jni jars
1c7c76476 Replaced obsolete comparator with builtin variant.
22c64be43 Cross platform fix for Java benchmark shell script.
81828592b Merge pull request #335 from fyrz/version-script-cross-platform
69d4c5123 Cross-platform fix version.sh
fcd13a77a Merge pull request #334 from fyrz/JavaDoc-Cleanup
bfb0246f5 Merge pull request #331 from fyrz/findbug_issues
05204bb11 Lint changes
da8ff9ff8 Fixed Findbugs issues
a5757ff3c Listing of changes
df3373fbf [Java] Fix compile error on DbBenchmark.java
4eb5a40f7 [Java] Fixed link error on library loading on Mac.
56dfd363f Fix a check in database shutdown or Column family drop during flush.
0e516a75d Fix lint errors in java/rocksjni/options.cc
8ea232b9e Add number of records dropped in compaction summary
f4086a88b perf_context.get_from_output_files_time is set for MultiGet() and ReadOnly DB too.
e869fc6a8 remove proper javadoc directory
c1273533b Merge pull request #333 from nbougalis/cleanups
99744e0c4 bump version to 3.6
45d526e22 singular javadoc directory
378f321da merge master to resolve merge conflicts
a1d3f0d2b don't fail if javadocs diretory doesn't exist
c832f1644 add not about updating pom version and rename pom to be unversioned
a213971d8 Don't return (or dereference) dangling pointer
2a1add673 use proper major/minor/micro version rather than hard coding 3.5.0
2d72f7807 update release docs in java
8322cf000 use javadoc instead of javadocs
079a612b5 Fix build_tools/version.sh
2e8012498 add javadoc and sources targets for sonatype
df08a2d03 add single rocksdbjni pom
deefcf476 make fat jar unclassified to satisfy sonatype
fd2545c80 add maven publication target and instructions
1e9af10ef Merge pull request #330 from fyrz/setFilterPolicyForBBTConfig
017354177 FilterTest
d410b39d5 BlockBasedTableConfig Filter policy support RocksJava
0908ddcea Don't keep managing two rocksdb version
d0916f452 add major minor micro version to java jars
d6987216c Merge pull request #327 from dalgaaf/wip-da-SCA-20141001
25888ae00 Merge pull request #329 from fyrz/master
89833e5a8 Fixed signed-unsigned comparison warning in db_test.cc
fcac705f9 Fixed compile warning on Mac caused by unused variables.
b3343fdea resolution for java build problem introduced by 5ec53f3edf62bec1b690ce12fb21a6c52203f3c8
187b29938 ForwardIterator: update prev_key_ only if prefix hasn't changed
5ec53f3ed make compaction related options changeable
d122e7bcf Update INSTALL.md
9d6f38086 backupable_db_test.cc: pass const string param by reference
8ff0b4095 document_db_test.cc: pass const string param by reference
177caca42 ttl/ttl_test.cc: pass const string param by reference
4a171882d db/version_set.cc: remove unnecessary checks
bf3bfd044 util/cache_test.cc: use static_cast over C-Style cast
86e29f033 document_db.cc: remove unused variable
28a6e3158 table/block_based_table_builder.cc: remove unused variable
091153493 db/db_test.cc: remove unused variable
6b6cedbb1 table/format.cc: reduce scope of some variables
55652043c table/cuckoo_table_reader.cc: pass func parameter by reference
5abd8add7 db/deletefile_test.cc: remove unused variable
d6483af87 db/db_test.cc: reduce scope of some variables
44cca0cd8 db/db_iter.cc: remove unused variable
986dad025 Merge pull request #324 from dalgaaf/wip-da-SCA-20140930
3a0d498a3 rebase master
8ee75dca2 db/memtable.cc: remove unused variable merge_result
0fd8bbca5 db/db_impl.cc: reduce scope of prefix_initialized
676ff7b1f compaction_picker.cc: remove check for >=0 for unsigned
e55aea551 document_db.cc: fix assert
d517c8364 in_table_factory.cc: use correct format specifier
b14037556 ttl/ttl_test.cc: prefer prefix ++operator for non-primitive types
43c789c8f spatialdb/spatial_db.cc: use !empty() instead of 'size() > 0'
0de452ee9 document_db.cc: pass const parameter by reference
4cc8643ba util/ldb_cmd.cc: prefer prefix ++operator for non-primitive types
af8c2b2d9 util/signal_test.cc: suppress intentional null pointer deref
33580fa39 db/db_impl.cc: fix object handling, remove double lines
873f1356a db_ttl_impl.h: pass func parameter by reference
855845714 ldb_cmd_execute_result.h: perform init in initialization list
063471bf7 table/table_test.cc: pass func parameter by reference
93548ce8f table/cuckoo_table_reader.cc: pass func parameter by ref
b8b7117e9 db/version_set.cc: use !empty() instead of 'size() > 0'
8ce050b51 table/bloom_block.*: pass func parameter by reference
53910ddb1 db_test.cc: pass parameter by reference
68ca53416 corruption_test.cc: pass parameter by reference
726ac5bca shrink vagrant commands to single line
a2f98ef61 fix tabs in Makefile
7506198da cuckoo_table_db_test.cc: add flush after delete
1f963305a Print MB per second compaction throughput separately for reads and writes
ffe3d490d Add an instruction about SSE in INSTALL.md
0b923f0f9 add centos 5.6 build instead of ubuntu.
ee1f3ccb0 Package generation for Ubuntu and CentOS
f0f795549 Fixing comile errors on OS X
99fb613e5 remove 2 space linter
b2d64a486 Fix linters, second try
747523d24 Print per column family metrics in db_bench
56ebd4087 Fix arc lint (should fix#238)
637f89179 Merge pull request #321 from eonnen/master
827e31c74 Make test use a compatible type in the size checks.
fd5d80d55 CompactedDB: log using the correct info_log
2faf49d5f use GetContext to replace callback function pointer
6a64ea617 add note about java 7
983d2de2d Add AUTHORS file. Fix#203
c4519c777 fix mis-named jar in JNI loader
abd70c5e1 Merge pull request #316 from fyrz/ReverseBytewiseComparator
2dc6f62bb handle kDelete type in cuckoo builder
8b8011a68 Changed name of ReverseBytewiseComparator based on review comment
b8e26615a since we're not sharing folders with the vm, copy built .so files and jars back to host system.
389edb6b1 universal compaction picker: use double for potential overflow
4e735bb7f Rsync files to VM rather than sync folders, since sync folders was causing clock skew and confusig make.
82a8f43cc Document RELEASE.mdgit status
9db13987b Update RocksDB's Java bindings to support multiple native RocksDB builds in the same Jar file. Cross build RocksDB for linux32 and linux64 using Vagrant. Build a cross-platform fat jar that contains osx, linux32, and linux64 RocksDB static builds.
534048426 Built-in comparator(s) in RocksJava
d439451fa delay initialization of cuckoo table iterator
94997eab5 reduce memory usage of cuckoo table builder
c6275956e improve memory efficiency of cuckoo reader
581442d44 option to choose module when calculating CuckooTable hash
fbd2dafc9 CompactedDBImpl::MultiGet() for better CuckooTable performance
3c6800610 CompactedDBImpl
f7375f39f Fix double deletes
21ddcf6e4 Remove allow_thread_local
fb4a492cc Merge pull request #311 from ankgup87/master
611e286b9 Merge branch 'master' of https://github.com/facebook/rocksdb
0103b4498 Merge branch 'master' of ssh://github.com/ankgup87/rocksdb
1dfb7bb98 Add block based table config options
cdaf44f9a Enlarge log size cap when printing file summary
7cc1ed7f0 Merge pull request #309 from naveenatceg/staticbuild
ba6d660f6 Resolving merge conflict
51eeaf65e Addressing review comments
fd7d3fe60 Addressing review comments (adding a env variable to override temp directory)
cf7ace886 Addressing review comments
0a29ce539 re-enable BlockBasedTable::SetupForCompaction()
55af37075 Remove TODO for checking index checksums
3d74f0997 Fix compile
53b003995 Fix release compile
d0de413f4 WriteBatchWithIndex to allow different Comparators for different column families
57a32f147 change target_file_size_base to uint64_t
5e6aee432 dont create backup_input if compaction filter v2 is not used
49b5f94c5 Merge pull request #306 from Liuchang0812/fix_cast
787cb4db2 remove cast, replace %llu with % PRIu64
a7574d4fa Update logging.cc
7e0dcb953 Update logging.cc
57fa3cc5b Merge pull request #304 from Liuchang0812/fix-check
cd44522a9 Merge pull request #305 from Liuchang0812/fix-logging
6a031b6a8 remove unused variable
4436f17bd fixed#303: replace %ld with % PRId64
7a1bd057f Merge pull request #302 from ankgup87/master
423e52cd4 Merge branch 'master' of https://github.com/facebook/rocksdb
bfeef94d3 Add rate limiter
ed9a2df8c fix unity build
32f2532a0 Print compression_size_percent as a signed int
976caca09 Skip AllocateTest if fallocate() is not supported in the file system
3b897cddd Enable no-fbcode RocksDB build
f44594743 RocksDB: Format uint64 using PRIu64 in db_impl.cc
e17bc65c7 Merge pull request #299 from ankgup87/master
b93797abc Fix build
adae3ca1f [Java] Fix JNI link error caused by the removal of options.db_stats_log_interval
90b8c07b4 Fix unit tests errors
51af7c326 CuckooTable: add one option to allow identity function for the first hash function
035043559 Fixed a signed-unsigned comparison in spatial_db.cc -- issue #293
2fb1fea30 Fix syncronization issues
6f0964e37 Only run make unity on travis instead of make check
9ed1b49a2 Build unity build on make check
ff7689561 Remove some unnecessary constructors
feadb9df5 fix cuckoo table builder test
3c232e164 Fix mac compile
54cada92b Run make format on PR #249
27b22f13a Merge pull request #249 from tdfischer/decompression-refactoring
fb6456b00 Replace naked calls to operator new and delete (Fixes#222)
5600c8f6e cuckoo table: return estimated size - 1
a062e1f2c SetOptions() for memtable related options
e4eca6a1e Options conversion function for convenience
a7c209452 Merge pull request #292 from saghmrossi/master
4d0523452 Merge branch 'master' of github.com:saghmrossi/rocksdb
60a4aa175 Test use_mmap_reads
94e43a1df [Java] Fixed 32-bit overflowing issue when converting jlong to size_t
f9eaaa66e added include for inttypes.h to fix nonworking printf statements
f090575e4 Replaced "built on on earlier work" by "built on earlier work" in README.md
faad439ac Fix#284
49aacd8d2 Fix make install
acb9348ff [Java] Include WriteBatch into RocksDBSample.java, fix how DbBenchmark.java handles WriteBatch.
4a27a2f19 Don't sync manifest when disableDataSync = true
9b8480d93 Merge pull request #287 from yinqiwen/rate-limiter-crash-fix
28be16b1d fix rate limiter crash #286
04ce1b25f Fix#284
add22e351 standardize scripts to run RocksDB benchmarks
dee91c259 WriteThread
540a257f2 Fix WAL synced
24f034bf4 Merge pull request #282 from Chilledheart/develop
49fe329e5 Fix build issue under macosx
ebb5c65e6 Add make install
0352a9fa9 add_wrapped_bloom_test
9c0e66ce9 Don't run background jobs (flush, compactions) when bg_error_ is set
a9639bda8 Fix valgrind test
d1f24dc7e Relax FlushSchedule test
3d9e6f775 Push model for flushing memtables
059e584dd [unit test] CompactRange should fail if we don't have space
dd641b211 fix RocksDB java build
53404d9fb add_qps_info_in cache bench
a52cecb56 Fix Mac compile
092f97e21 Fix comments and typos
6cc12860f Added a few statistics for BackupableDB
0a42295a2 Fix SimpleWriteTimeoutTest
06d986252 Always pass MergeContext as pointer, not reference
d343c3fe4 Improve db recovery
6bb7e3ef2 Merger test
88841bd00 Explicitly cast char to signed char in Hash()
52311463e MemTableOptions
1d284db21 Addressing review comments
55114e7f4 Some updates for SpatialDB
171d4ff4a remove TailingIterator reference in db_impl.h
9b0f7ffa1 rename version_set options_ to db_options_ to avoid confusion
2d57828d0 Check stop level trigger-0 before slowdown level-0 trigger
659d2d50c move compaction_filter to immutable_options
048560a64 reduce references to cfd->options() in DBImpl
011241bb9 DB::Flush() Do not wait for background threads when there is nothing in mem table
a2bb7c3c3 Push- instead of pull-model for managing Write stalls
0af157f9b Implement full filter for block based table.
9360cc690 Fix valgrind issue
02d5bff39 Merge pull request #277 from wankai/master
88a2f44f9 fix comments
7c16e3922 Merge pull request #276 from wankai/master
823773837 replace hard-coded number with named variable
db8ca520b Merge pull request #273 from nbougalis/static-analysis
b7b031f42 Merge pull request #274 from wankai/master
4c2b1f097 Merge remote-tracking branch 'upstream/master'
a5d286307 typo improvement
9f8aa0939 Don't leak data returned by opendir
d1cfb71ec Remove unused member(s)
bfee319fb sizeof(int*) where sizeof(int) was intended
d40c1f742 Add missing break statement
2e97c3898 Avoid off-by-one error when using readlink
40ddc3d6c add cache bench
9f1c80b55 Drop column family from write thread
8de151bb9 Add db_bench with lots of column families to regression tests
c9e419ccb rename options_ to db_options_ in DBImpl to avoid confusion
5cd0576ff Fix compaction bug in Cuckoo Table Builder. Use kvs_.size() instead of num_entries in FileSize() method.
0fbb3facc fixed memory leak in unit test DBIteratorBoundTest
adcd2532c fix asan check
4092b7a0b Merge pull request #272 from project-zerus/patch-1
bb6ae0f80 fix more compile warnings
6d3144118 Merge pull request #271 from nbougalis/cleanups
0cd0ec4fe Plug memory leak during index creation
4329d74e0 Fix swapped variable names to accurately reflect usage
45a5e3ede Remove path with arena==nullptr from NewInternalIterator
5665e5e28 introduce ImmutableOptions
e0b99d4f5 created a new ReadOptions parameter 'iterate_upper_bound'
51ea88900 Fix travis builds
a4816269f Relax backupable rate limiting test
f7f973d35 Merge pull request #269 from huahang/patch-2
ef5b38472 fix a few compile warnings
2fd3806c8 Merge pull request #263 from wankai/master
1785114a6 delete unused Comparator
1b1d9619f update HISTORY.md
703c3eacd comments about the BlockBasedTableOptions migration in Options
4b5ad8865 Merge pull request #260 from wankai/master
19cc588b7 change to filter_block std::unique_ptr support RAII
9b976e34f Merge pull request #259 from wankai/master
5d25a4693 Merge remote-tracking branch 'upstream/master'
dff2b1a8f typo improvement
343e98a7d Reverting import change
ddb8039e3 RocksDB static build Make file changes to download and build the dependencies .Load the shared library when RocksDB is initialized
REVERT: 1fdd726a8 Hotfix RocksDB 3.5
REVERT: d67500a59 Add `make install` to Makefile in 3.5.fb.
REVERT: 4cb631aa0 update HISTORY.md
REVERT: cfd0946be comments about the BlockBasedTableOptions migration in Options
git-subtree-dir: src/rocksdb2
git-subtree-split: aead4041720f40e305e7f9d432875749d91d5a2d
Previously if you mistyped the "submit_multisigned" command as
"submit_multisign", the returned message was "Internal error". Not
very helpful. It turns out this was caused by a small amount of
code in RPCCall.cpp. Removing that code improves two situations:
1. It improves the situation with a mistyped command. Now the
command returns "Unknown method" and provides the string of
the mistyped command.
2. The "transaction_entry", if properly entered in its command
line form, would fire an assert. That assert is now removed.
In the process, it was discovered that the command line form of
the "transaction_entry" command has not worked correctly for at
least a year. Therefore support for that the command line form
of "transaction_entry" is added along with appropriate unit
tests.
It is common for a validator operator to connect their validator
only to other nodes under their control, using clustering. Relaying of
untrusted validations and proposals must be unreliable to prevent
denial of service attacks. But currently, they are unreliable even
within a cluster.
With this change, a cluster member's decisison to relay (or originate)
a validation or proposal is honored by other cluster members. This
ensures that validators in a cluster will get reliable relaying to
hubs outside the cluster, even if other members of the cluster do not
have that validator on their UNL.
* Changes to Travis seem to be making these builds slower, and are
resulting in spurious timeouts. Remove the builds that fail more
often than not until we complete our migration to another build
system.
* Use CMake for unity builds, instead of the slow build.
* Can be exercised from the command line with json2
* Rewrite Env::do_rpc to call the same code as
rpc from the command line. This puts rpc
handling logic in one place.
* If any of the destructor, copy assignment or copy constructor
are user-declared, both copy members should be user-declared,
otherwise the compiler-generation of them is deprecated.
* If any of the destructor, copy assignment or copy constructor
are user-declared, both copy members should be user-declared,
otherwise the compiler-generation of them is deprecated.
* Remove composite helper functions
* Add set difference and Bitset/uint256 operators
* Convert tests to use new feature bitset set difference operator
In order to automatically run unit tests with newly created
amendments, prefer to start with jtx::supported_features() and
then subtract unwanted features.
These changes identified a few bugs that were hiding in
amendments. One of those bugs, in FlowCross, is not yet fixed.
By uncommenting the test in CrossingLimits_test.cpp you can see
failures relating to that bug. Since FlowCross is not yet
enabled on the network we can fix the bug at our convenience.
Both Tickets and SHAMapV2 have been around for a while and don't
look like they will be enabled on the network soon. So they are
removed from the supportedAmendments list. This prevents Env
from automatically testing with Tickets or SHAMapV2 enabled,
although testing with those features can still be explicitly
specified.
Drive-by cleanups:
o supportedAmendments() returns a const reference rather than
a fresh vector on each call.
o supportedAmendments() implementation moved from Amendments.cpp
to Feature.cpp. Amendments.cpp deleted.
o supportedAmendments() declared in Feature.h. All other
declarations deleted.
o preEnabledAmendments() removed, since it was empty and only
used in one place. It will be easy to re-add when it is needed.
o jtx::all_features_except() renamed to
jtx::supported_features_except(), which is more descriptive.
o jtx::all_amendments() renamed to jxt::supported_amendments()
o jtx::with_features() renamed to with_only_features()
o Env_test.cpp adjusted since featureTickets is no longer
automatically enabled for unit tests.
- Separate `Scheduler` from `BasicNetwork`.
- Add an event/collector framework for monitoring invariants and calculating statistics.
- Allow distinct network and trust connections between Peers.
- Add a simple routing strategy to support broadcasting arbitrary messages.
- Add a common directed graph (`Digraph`) class for representing network and trust topologies.
- Add a `PeerGroup` class for simpler specification of the trust and network topologies.
- Add a `LedgerOracle` class to ensure distinct ledger histories and simplify branch checking.
- Add a `Submitter` to send transactions in at fixed or random intervals to fixed or random peers.
Co-authored-by: Joseph McGee
We need to ensure that pointers and/or references to existing
elements will not be invalidated during the course of element
insertion and removal.
Containers like std::vector do not offer this guarantee, and
cannot be used as the underlying container for the stack.
By choosing to explicitly specify std::deque as the underlying
cotnainer, we avoid:
- the unlikely possibility of the C++ standards committee
changing the default template parameter to a container;
- the more likely possibility of an accidental change by
a programmer, without fully considering the consequences.
* Stores recent history of "good" ledgers. Uses the maximum as the
expected ledger size. When a large value drops off, use a 90%
backoff to go down to to the new maximum.
* If consensus is unhealthy, wipe the history in addition to the current
clamping.
* Include .md doc files in xcode and VS projects
In support of dynamic validator list, this changeset:
1. Adds a new `validator_list_expires` field to `server_info` that
indicates when the current validator list will become stale.
2. Adds a new admin only `validator_lists` RPC that returns the
current list of known validators and the most recent published validator
lists.
3. Adds a new admin only `validator_sites` RPC that returns the list of
configured validator publisher sites and when they were most recently
queried.
Manifests of validators newly added to a published validator list are
not reliably propagated to network nodes.
This solves the problem by allowing a published validator list to
include the manifest.
RIPD-1559
2017-11-22 16:25:40 -06:00
3670 changed files with 230061 additions and 948258 deletions
# No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
# Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.