With the rapid development of the Internet, all walks of life have higher and higher requirements for Internet services. How much business data can the service architecture support? Can the speed of service response meet the requirements? Our architects think about these issues every day.
For services such as databases or object storage, they are limited by their inherent design goals and often do not have very good performance, with response times of seconds. This is where high-performance caching is needed to speed up our services, with response times of milliseconds or even less than 1ms.
The cache service needs to be set in front of other services. The client accesses the cache first to query its own data, and accesses the actual service only when the data the client needs does not exist in the cache. Data retrieved from the actual service is stored in the cache for later use.
Caching is designed to be as fast as possible, but it causes other problems. For example, Memcached and Redis are widely used in the industry. They are in-memory caches. The maximum capacity of a node cannot exceed the memory of the entire system.
Once the server is restarted, Memcached is completely lost; Redis is a little better, but it also takes quite a bit of time to read back into memory from data files on disk.
When we decided to write a caching service in Go, the first thing that came to mind was the HTTP service. Because it is really convenient to write http-based cache service in Go language, we only need a map to save data, write a handler to handle the request, then call HTTP.ListenAndServe, and finally run with Go Run. It’s as simple as that. You don’t need to worry about complicated concurrency or design your own network protocol. Go’s HTTP framework takes care of the bottom line.
What we will implement in this article is a simple memory caching service, where all cached data is stored in the server’s memory. Once the server is restarted, all data is zeroed out.
The interface of the cache service
1.1.1 REST interface
The interface in this chapter supports three basic operations: Set, Get, and Del. It also supports querying the status of the cache service. The Set operation is used to Set a key value pair to the cache server through the HTTP PUT method. The Get operation is used to query a key and obtain its value through the HTTP Get method. The Del operation is used to remove a key from the cache through HTTP’s DELETE method. The state of the cache service we can query includes how many key-value pairs are currently cached, how many bytes are occupied by all the keys, and how many bytes are occupied by all the values.
The client uses the HTTP PUT method to set a pair of key-value pairs to the cache server, which stores the key-value pairs in a map created on the memory heap.
Here /cache/ is a URL that identifies the location of the cached value. URL is an abbreviation for Uniform Resource Locator. It is a network address that refers to the location of a network Resource on the network. The HTTP Request body contains the value corresponding to the key.
The client obtains the value of the key from the cache server using the HTTP GET method. The server searches for the key in the map. If the key does NOT exist, the server returns an HTTP error code 404 NOT FOUND. If the key exists, the server returns the corresponding value in the HTTP response Body.
The client obtains the value of the key from the cache server using the HTTP GET method. The server searches for the key in the map. If the key does NOT exist, the server returns an HTTP error code 404 NOT FOUND. If the key exists, the server returns the corresponding value in the HTTP response Body.
The client deletes the key from the cache using the HTTP DELETE method. Whether or not the key existed before, it will not exist after, and the server always returns the HTTP error code 200 OK.
The client gets the state of the cache service through this interface, and the state returned in the HTTP response body is a cache.stat structure encoded in JSON format (see Examples 1-3).
1.1.2 Cache Set process
We can summarize the Set process in a simple diagram, as shown in Figure 1-1.
Figure 1-1 In Memory cache Set process
The client PUT request provides a key and a value. CacheHandler implements the HTTP. Handler interface, whose ServeHTTP method parses HTTP requests and calls the Set method of the cache. cache interface.
In the cache module, the inMemoryCache structure implements the cache interface, and its Set method ultimately saves the key-value pairs in the memory map. The cacheHandler eventually returns an HTTP Error number to the client to indicate the result, 200 OK if successful and 500 Internal Server Error otherwise.
A map, like a map in most modern programming languages, is a hash table data structure used to hold key-value pairs. Keys can be queried and set using brackets [].
Because the program hashes and masks the key to directly obtain the offset of the storage key, it can achieve a query and setup complexity near O(1). We say almost O(1) because it is possible that the two keys will have the same offset after hashing and masking, in which case we will have to continue our linear search, but the probability of that happening is very small.
1.1.3 Caching the Get Process
Figure 1-2 shows the cache Get process.
Figure 1-2 IN Memory cache Get process
The client’s Get request provides the key. The cacheHandler ServeHTTP method parses HTTP requests and calls the Cache. cache interface’s Get method. The Get method of the inMemoryCache structure queries the map for the value corresponding to the key and returns the value. CacheHandler writes value to the HTTP response body and returns 200 OK. If cache.cache. Get returns an Error, cacheHandler returns 500 Internal Server Error. If the value length is 0, the key does Not exist and the cacheHandler returns 404 Not Found.
1.1.4 Cache Del Process
Figure 1-3 shows the PROCESS for caching Del.
Figure 1-3 IN Memory cache Del process
The client’s DELETE request provides a key. The ServeHTTP method of the cacheHandler parses the HTTP request and calls the Del method of the cache. cache interface. The Del method of the inMemoryCache structure queries the map for the presence of a key and, if so, calls the delete function to delete the key. CacheHandler returns 500 Internal Server Error if the cache.cache.del method returns an Error, and 200 OK otherwise.
With REST interfaces and processes covered, let’s look at how to implement them.
Go language implementation
1.2.1 Implementation of main package
The main package of the cache service has only one function, the main function. In Go, if a project needs to be compiled as an executable, its source code needs to have a main package with a main function that acts as an entry function for the executable. If a project does not need to be compiled as an executable, but simply implements a library, you can do without the main package and main functions. Our cache service needs to be compiled into an executable program, so we need to provide the main package and main function. See Example 1-1 for the implementation of main:
Case 1-1 the main function
Our main function is very simple. All it needs to do is call cache.New to create a New instance of the cache. cache interface, c, and then call HTTP. New with c to create a pointer to the http.Server structure and call its Listen method.
Cache. New specifies that the New function we call belongs to the cache package. Go calls to functions in the same package do not require the package name in front of the function, and the Go compiler will look it up in the current package by default. Calling a function in another package requires specifying the package name so that the Go compiler knows where to look for the function. Here we are calling the New function of the cache package from the main package, so we need to specify the package name.
1.2.2 Implementation of cache packet
We implement caching of services in cache packages. In the cache package, we first declare a cache interface, as shown in Example 1-2.
Case 1-2 cache interface
In Go, the interface and implementation are completely separate. The interface even has its own type interface. Developers are free to declare an interface and then implement it in one or more ways. In example 1-2, we see an interface declaration called Cache.
Within an interface, we declare methods, and an interface is a collection of methods within that interface. Any structure that implements all of the methods declared by an interface is considered to implement that interface. An interface can be implemented in more than one structure, which means that the same interface can be implemented in many ways, which is how the Go language implements polymorphism.
Our Cache interface declares four methods: Set, Get, Del, and GetStat.
The Set method is used to Set a key-value pair in the cache. It takes two arguments of type string and []byte, where string is the type of key and []byte is the type of value. The parentheses before byte indicate that the type is a slice of byte. The internal implementation of a slice in Go can be thought of as an address pointing to the first element of the slice and the length of that slice. A slice differs from an Array in that the length of an Array is fixed, whereas a slice is a view of the underlying Array whose length can be dynamically adjusted. The Set method returns only one value. If the return value is of type error, it is used to return the error of the Set operation, and nil when the Set operation succeeds.
The Get method retrieves value from the cache based on key, so it takes a string argument and returns two values, []byte and error. In the Go language, when a function has multiple return values, you need to enclose them in parentheses ().
The Del method removes the key from the cache, so it has only a string argument and an error return value.
The GetStat method is used to get the status of the cache. It takes no arguments and only returns a Stat value. Stat is a structure, as shown in Examples 1-3.
Example 1-3 Associated implementation of the STAT structure
Go programming does not simply declare type interfaces; it must also implement interfaces. The implementation of the interface needs to be attached to a Type struct. Stat is a structure with three internal fields. Count indicates the number of key/value pairs currently held by the cache. KeySize and ValueSize indicate the total number of bytes occupied by the key and value, respectively.
Structures can also contain methods, unlike interfaces where structures must implement these methods and interfaces only need to be declared. The Stat structure implements the add and del methods, which change the state of the cache when a new key-value pair is added and a key-value pair is deleted, respectively.
Now that we know the full Cache interface, we can look at the implementation of the New function, as shown in Examples 1-4.
Example 1-4New function implementation
The New function of the cache package is used to create and return a cache interface. It takes a string parameter typ, which specifies the structure type of the cache interface to be created.
We declare a variable C of type Cache interface in the first line of the function body. When tyP string is equal to “inmemory”, we assign the return value of newInMemoryCache to C. If C is nil, we call Panic and exit the whole program, otherwise we print a log telling the cache to start service and return C.
The Cache service implemented in this paper is a kind of in memory Cache. The structure that implements the Cache interface is named inMemoryCache, as shown in Example 1-5.
Example 1-5 InmemoryCache code
The inMemoryCache structure contains a member C, a map of type STRING as key and []byte as value, used to store key-value pairs. A mutex, of type sync.RWMutex, is used to provide read/write lock protection for concurrent access to the map. A Stat to record cache status.
A Go map can be read by multiple Goroutines at the same time, but it cannot be written or read by multiple Goroutines at the same time. Therefore, we must use a read/write lock to protect the concurrent read/write of the map. When multiple Goroutines read at the same time, they will call mutex.rlock ().
When at least one goroutine needs to write, it calls mutex.lock (), at which point it waits for all other read-write locks to be released, and then locks itself. After it locks, other Goroutines that need to Lock must wait for it to unlock. The type of the read/write lock mutex is sync.RWMutex. Sync is a standard package of Go language, which provides mutex, RWMutex and other mutex implementation.
Of particular note is Stat, which is a Stat structure of type but does not provide member names, which is called inline in the Go language. A structure can be embedded with multiple structures and interfaces, or only multiple interfaces.
The Go language implements inheritance through embedding, which can be thought of as the parent of the outer structure/interface. All members/methods of an embedded structure/interface can be accessed directly through the outer structure/interface. The first letter of those members/methods does not need to be capitalized. (Normally, we can only access members/methods with uppercase letters from outside a structure; members/methods that access their own embedded members are exempt from this restriction.) When we need to access an embedded member itself, we can refer to it directly by its type, as we do in the inmemoryCache.getstat function.
1.2.3 Implementation of HTTP package
HTTP packages are used to implement our HTTP service functionality. Since there is no need to use polymorphism, we do not declare an interface in the HTTP package, but instead declare a Server structure directly, as shown in Examples 1-6.
Example 1-6Server related implementation
The Server structure contains cache. cache, which is the cache interface of the cache package. The embedding of this interface in the Server structure of the HTTP package means that http.Server also implements the cache. cache interface in a manner that is determined by the actual embedded structure.
Next we see that the Server’s Listen method calls http.Handle, which registers two handlers to Handle the HTTP endpoints of /cache/ and /status.
The important thing to note here is that the http.Handle function is not part of our HTTP package, but the Go language’s own NET/HTTP standard package. Remember? The Server structure itself is in our HTTP package and references to its own package name do not need to specify the package name, so when we specify the HTTP package name, the Go language compiler knows to look for the name in the NET/HTTP package.
The server. cacheHandler method returns an HTTP. Handler interface that handles requests from HTTP endpoints /cache/, namely, the Set, Get, and Del operations of the cache, as shown in Examples 1-7.
Example 1-7 Related implementation of cacheHandler
The cacheHandler structure is embedded with a pointer to the Server structure and implements the ServeHTTP method, which implements the HTTP.handler interface. Example 1-8 shows the definition of the Go standard package NET/HTTP for the Handler interface.
Example 1-8 Definition of the Handler interface in the GO standard package NET/HTTP
The ServeHTTP method of cacheHandler parses the URL to GET the key and decides to call the Set/ GET/ Del methods of cache. cache based on the three ways to PUT/GET/DELETE HTTP requests.
Here we see a higher-order use of Go embedding — multiple embedding: cacheHandler has a pointer to the Server structure embedded in it, while Server has the cache.cache interface embedded in it. CacheHandler can then access cache.cache methods directly.
The server.statusHandler method also returns an HTTP. Handler interface, which is implemented in Examples 1-9.
Example 1-9 StatusHandler implementation
Like cacheHandler, statusHandler inlays a pointer to the Server structure and implements ServeHTTP methods. This method calls the GetStat method of cache. cache and encodes the returned cache.Stat structure into a byte slice B in JSON format, which is written to the HTTP response body.
If you’re a programmer, you might have a question in your mind when you see this. Are we getting too complicated to implement this way? To handle requests from two HTTP endpoints, we need to implement two Handler constructs and implement their ServeHTTP methods separately. Can we implement ServeHTTP methods directly on the Server constructs and differentiate HTTP requests based on URLS?
This is implementationally possible, but it means that the Server’s Server HTTP has two different responsibilities, handling two types of HTTP requests. Separating these two types of requests into different structures is consistent with SOLID’s single responsibility principle.
With the implementation of the Go language introduced, we need to put the program up and running and perform functional tests to verify our implementation.
Distributed Caching — Principles, Architecture, and Go Language Implementation
The Hu Shijie
Click here to buy paper books
The book is divided into three parts, each with three chapters. The first part is the realization of basic functions, mainly introduces the IN Memory caching service based on HTTP, HTTP/REST protocol, TCP and so on. In part 2, we will focus on improving the performance of cache services from various aspects, including the principle of pipeline, RocksDB batch write, etc. The last part is about HE distributed cache service cluster, mainly introduces distributed cache cluster, node rebalance function, etc.