These reviews

  • LevelDB Fully resolved (0) : Rationale and overall architecture
  • LevelDB fully parse (1) : MemTable
  • LevelDB fully resolved (2) : Log
  • LevelDB fully resolved (3) : SSTable

In content, the Manifest file holds metadata for the entire LevelDB instance, such as the sstables for each level.

In format, the Manifest file is actually a log file, and a log record is a VersionEdit.

VersionEdit

LevelDB represents a metadata change using VersionEdit. The Manifest file holds VersionEdit serialized data. LevelDB metadata changes include:

std: :string comparator_;
uint64_t log_number_;
uint64_t prev_log_number_;
uint64_t next_file_number_;
SequenceNumber last_sequence_;

std: :vector<std::pair<int, InternalKey> > compact_pointers_;
DeletedFileSet deleted_files_;
std: :vector<std::pair<int, FileMetaData> > new_files_;
Copy the code
  1. Comparator_ : Name of the comparator. This was specified when LevelDB was created and cannot be changed.
  2. Log_number_ : indicates the minimum valid log number. Log files smaller than log_numbers_ can be deleted.
  3. Prev_log_number_ : Deprecated, code reserved for compatibility with older versions of LevelDB.
  4. Next_file_number_ : indicates the number of the next file.
  5. Last_sequence_ : Indicates the maximum sequence number in the SSTable.
  6. Compact_pointers_ : Records the start key for each layer to conduct the next compaction.
  7. Deleted_files_ : Indicates the SSTable that can be deleted (level-no > file-no).
  8. New_files_ : new SSTable.

VersionEdit serializes and deserializes through the member functions EncodeTo and DecodeFrom. A VersionEdit serialized version with all fields filled in would have the following format (note that not every VersionEdit consists of the contents of all of the following fields, usually only a portion) :

kComparator comparator_
kLogNumber log_number_
kPrevLogNumber prev_log_number_
kNextFileNumber next_file_number_
kLastSequence last_sequence_
kCompactPointer level internal_key kCompactPointer level internal_key ...
kDeletedFile level fileno kDeletedFile level fileno ...
kNewFile level fileno file-size smallest largest kNewFile level fileno file-size smallest largest ...
Copy the code

Version

Version is the database state obtained after VersionEdit is applied — which SStables are included in the current Version, and is secured by reference counting for multi-threaded concurrent access. Before reading the SSTable, you need to call Version::Ref to increase the reference count. If the SSTable is not used, you need to call Version::UnRef to decrease the reference count.

VersionSet

VersionSet is a collection of versions.

As the state of the database changes, VersionEdit is constantly generated within LevelDB — which in turn generates new versions. At this point, the old Version may still be used by the ongoing request. Therefore, multiple versions may exist at the same time.

VersionSet These versions are maintained in a linked list and a node (AppendVersion) is inserted at the end of each Version generated.

More details will be provided when this Compaction happens.