[Rocksdb] Support Merge operation in rocksdb

Summary:
This diff introduces a new Merge operation into rocksdb.
The purpose of this review is mostly getting feedback from the team (everyone please) on the design.

Please focus on the four files under include/leveldb/, as they spell the client visible interface change.
include/leveldb/db.h
include/leveldb/merge_operator.h
include/leveldb/options.h
include/leveldb/write_batch.h

Please go over local/my_test.cc carefully, as it is a concerete use case.

Please also review the impelmentation files to see if the straw man implementation makes sense.

Note that, the diff does pass all make check and truly supports forward iterator over db and a version
of Get that's based on iterator.

Future work:
- Integration with compaction
- A raw Get implementation

I am working on a wiki that explains the design and implementation choices, but coding comes
just naturally and I think it might be a good idea to share the code earlier. The code is
heavily commented.

Test Plan: run all local tests

Reviewers: dhruba, heyongqiang

Reviewed By: dhruba

CC: leveldb, zshao, sheki, emayanke, MarkCallaghan

Differential Revision: https://reviews.facebook.net/D9651
This commit is contained in:
Haobo Xu
2013-03-21 15:59:47 -07:00
parent 37e97b1297
commit 05e8854085
37 changed files with 1209 additions and 123 deletions

View File

@@ -7,7 +7,8 @@
// count: fixed32
// data: record[count]
// record :=
// kTypeValue varstring varstring |
// kTypeValue varstring varstring
// kTypeMerge varstring varstring
// kTypeDeletion varstring
// varstring :=
// len: varint32
@@ -20,6 +21,7 @@
#include "db/memtable.h"
#include "db/write_batch_internal.h"
#include "util/coding.h"
#include <stdexcept>
namespace leveldb {
@@ -34,6 +36,10 @@ WriteBatch::~WriteBatch() { }
WriteBatch::Handler::~Handler() { }
void WriteBatch::Handler::Merge(const Slice& key, const Slice& value) {
throw std::runtime_error("Handler::Merge not implemented!");
}
void WriteBatch::Clear() {
rep_.clear();
rep_.resize(kHeader);
@@ -68,6 +74,14 @@ Status WriteBatch::Iterate(Handler* handler) const {
return Status::Corruption("bad WriteBatch Delete");
}
break;
case kTypeMerge:
if (GetLengthPrefixedSlice(&input, &key) &&
GetLengthPrefixedSlice(&input, &value)) {
handler->Merge(key, value);
} else {
return Status::Corruption("bad WriteBatch Merge");
}
break;
default:
return Status::Corruption("unknown WriteBatch tag");
}
@@ -108,6 +122,14 @@ void WriteBatch::Delete(const Slice& key) {
PutLengthPrefixedSlice(&rep_, key);
}
void WriteBatch::Merge(const Slice& key, const Slice& value) {
WriteBatchInternal::SetCount(this, WriteBatchInternal::Count(this) + 1);
rep_.push_back(static_cast<char>(kTypeMerge));
PutLengthPrefixedSlice(&rep_, key);
PutLengthPrefixedSlice(&rep_, value);
}
namespace {
class MemTableInserter : public WriteBatch::Handler {
public:
@@ -118,6 +140,10 @@ class MemTableInserter : public WriteBatch::Handler {
mem_->Add(sequence_, kTypeValue, key, value);
sequence_++;
}
virtual void Merge(const Slice& key, const Slice& value) {
mem_->Add(sequence_, kTypeMerge, key, value);
sequence_++;
}
virtual void Delete(const Slice& key) {
mem_->Add(sequence_, kTypeDeletion, key, Slice());
sequence_++;