KV learning camp

I think IT is necessary to introduce the Talent Plan KV training camp. It is a set of open source distributed KV storage practice course launched by PingCAP Company. This course contains 4 sub-projects:

  1. Project 1Requires participants to complete a standaloneKV Server
  2. Project 2Based onRaftAlgorithm to implement distributed key – value database server
  3. Project 3Need to beIn the Project 2Support multiple based onRaftThe cluster
  4. Project 4Need to beProject 3Support distributed transactions on the basis of

If all the sub-projects of TINyKV can be realized and a distributed KV database can be produced, I believe this is also a great harvest. As far as I know, there is only one set of MIT 6.824 courses at present. Ping Cap launched this tinyKV course is equivalent to fill the gap in the domestic course.

Current to participate in this project I’m asking for is the Go to the Lab, perhaps the reader when you see this article may switch to other languages such as Rust, as to what kind of programming language to realize, of course, is not important, it is important to read parts in the paper, it’s as if you understand a building design drawings, Then language is a tool for building bricks 🧱.

  • Address: github.com/kvbase/raft…

Since the organizer requires participants not to directly post the answer code of each Project in any way, of course I think it is right to do so, but the organizer encourages participants to share some solution ideas about the realization of lab. In the evening, I will spend some time to finish the first lab. In this paper, I will write about how Project 1 went into the pit.

Standalone KV Server

The first lab is a typical example of TDD development, what is TDD if the reader does not know themselves go to Google. Project 1 is to implement the API interface reserved by the question maker based on BadgerDB as the storage engine, and then an official makefile is built in the root directory of the Project. Participants only need to make project1 to check the completion of Project 1.

BadgerDB is a local key-value database based on Log Structured Merge (LSM) Tree developed by Dgraph. IO and developed by Go.

To develop this project you need to have the Go Mod foundation, you don’t have to fix it yourself.

When you execute make Project1, you can see that a lot of exceptions are thrown because the unit test written by the official engineer did not pass. All you need to do is to implement the function in the API of all the unit test case calls under/tinyKV /kv/server/server_test.go.

  1. The first thing you need to implement
package standalone_storage

import (
    "github.com/pingcap-incubator/tinykv/kv/config"
    "github.com/pingcap-incubator/tinykv/kv/storage"
    "github.com/pingcap-incubator/tinykv/proto/pkg/kvrpcpb"
)

// StandAloneStorage is an implementation of `Storage` for a single-node TinyKV instance. It does not
// communicate with other nodes and all data is stored locally.
type StandAloneStorage struct {
    // Your Data Here (1).
}

func NewStandAloneStorage(conf *config.Config) *StandAloneStorage {
    // Your Code Here (1).
    return nil
}

func (s *StandAloneStorage) Start(a) error {
    // Your Code Here (1).
    return nil
}

func (s *StandAloneStorage) Stop(a) error {
    // Your Code Here (1).
    return nil
}

func (s *StandAloneStorage) Reader(ctx *kvrpcpb.Context) (storage.StorageReader, error) {
    // Your Code Here (1).
    return nil.nil
}

func (s *StandAloneStorage) Write(ctx *kvrpcpb.Context, batch []storage.Modify) error {
    // Your Code Here (1).
    return nil
}

Copy the code
  1. The second thing you need to achieve
package server

import (
	"context"
	"github.com/pingcap-incubator/tinykv/proto/pkg/kvrpcpb"
)

// The functions below are Server's Raw API. (implements TinyKvServer).
// Some helper methods can be found in sever.go in the current directory

// RawGet return the corresponding Get response based on RawGetRequest's CF and Key fields
func (server *Server) RawGet(_ context.Context, req *kvrpcpb.RawGetRequest) (*kvrpcpb.RawGetResponse, error) {
    // Your Code Here (1).
    return nil.nil
}

// RawPut puts the target data into storage and returns the corresponding response
func (server *Server) RawPut(_ context.Context, req *kvrpcpb.RawPutRequest) (*kvrpcpb.RawPutResponse, error) {
    // Your Code Here (1).
    // Hint: Consider using Storage.Modify to store data to be modified
    return nil.nil
}

// RawDelete delete the target data from storage and returns the corresponding response
func (server *Server) RawDelete(_ context.Context, req *kvrpcpb.RawDeleteRequest) (*kvrpcpb.RawDeleteResponse, error) {
    // Your Code Here (1).
    // Hint: Consider using Storage.Modify to store data to be deleted
    return nil.nil
}

// RawScan scan the data starting from the start key up to limit. and return the corresponding result
func (server *Server) RawScan(_ context.Context, req *kvrpcpb.RawScanRequest) (*kvrpcpb.RawScanResponse, error) {
    // Your Code Here (1).
    // Hint: Consider using reader.IterCF
    return nil.nil
}
Copy the code

The corresponding unit test file is tinyKV/KV /server/server_test.go. The official engineer of the test case has written it. You only need to implement the underlying code and run through the unit test to complete Project 1. Unit tests can be tested one by one, first comment out all unit tests, then implement a uncomment one, run one, good luck! .