GO Storage Storage encapsulation

Go Storage Encapsulates the Storage package

purpose

The purpose of packaging storage packages is to make it easier to store data in different media, either cloud storage or local storage, within a project. And to provide the object name detection, the authority of the judgment, is to reduce the user in the storage of some problems ignored, so as to avoid possible errors. In addition, by switching the parameters of the configuration file, you can switch the storage media without modifying the code.

If you have your own storage medium, implement the interface in stroage.go and call storage.Register.

Storage media

At present, provides the local storage (FS), Qingyun storage (QS), Tencent Cloud (COS) storage package.

Making the address

Github.com/navi-tt/sto…

Interface

type Storage interface {
	// Read and write a file as an object: Get and Put

	Init(cfg string) (Storage, error)

	// Save data to a file
	Put(key string, r io.Reader, contentLength int64) error

	// Write from one file to another file
	PutByPath(key string./* Take a path, relative or absolute */src string) error

	// Get the voice stream
	FileStream(key string) (io.ReadCloser, *FileInfo, error)

	// Get data
	Get(key string, wa io.WriterAt) error

	// Get a file
	GetToPath(key string./* Take a path, relative or absolute */dest string) error

	// Get the file size, modification time, and permission
	Stat(key string) (*FileInfo, error)

	// Delete files
	Del(key string) error

	// Get the file size
	Size(key string) (int64, error)

	// Check whether the file exists
	IsExist(key string) (bool, error)
}
Copy the code

Methods provided

In the case of local storage (FS), all the sample code is on Github above

Initialize the

storageType = fs
storageConf = {"BaseDir":"./testData"}
Copy the code

fsStorage, err := storage.Init(cfg.storageType,cfg.storageConf)
iferr ! =nil {
	fmt.Printf("err : %s\n", err.Error())
	return
}
Copy the code

To initialize local storage, call storage.init (). If the type is not an encapsulated storage type, an error is returned, indicating that the type is not registered to the storage.

The base dir parameter is mainly for folder prefix, because in most cases (depending on the project you are writing), the path prefix is saved the same, but is stored in a root directory with different folder categories. So with the base dir definition, you can automatically concatenate the base dir in subsequent operations without missing it. Of course, there is no problem with not writing.

Saves data to an object

Put(key string, r io.Reader, contentLength int64) error

The PUT method reads data from any readable object and saves it to the corresponding path. This method checks the permissions of the stored object and the name of the object so that it does not worry about returning an error that was returned by the system without knowing why.

buf := bytes.NewBuffer(nil)
buf.WriteString("this is test!")
if err := fsStorage.Put("test_fs.txt", buf, int64(buf.Len())); err ! =nil {
	fmt.Printf("fs put err : %s\n", err.Error())
	return
}
Copy the code

Read data from one object and store it in another object

PutByPath(key string, src string) error

This method reads data from an object and stores it to the corresponding path. This method checks if SRC exists and returns an error if it does not. The SRC address must be either a relative path or an absolute path. SRC does not spell base dir.

err := fsStorage.PutByPath("test_fs_put_by_path.txt".".. /testdata/test_fs.txt")
iferr ! =nil {
	return er
}
Copy the code

Get file stream

FileStream(key string) (io.ReadCloser, *FileInfo, error)

An empty error is returned if the object content is empty

fd,stat,err := fsStorage.FileStream("test_fs_put_by_path.txt")
iferr ! =nil {
	return err
}
Copy the code

Access to the object

Get(key string, wa io.WriterAt) error

The Get method takes data from an object and writes it to a writeAter.

fd, _ := os.OpenFile("test.txt", os.O_CREATE|os.O_RDWR|os.O_APPEND, 0766)
err := fsStorage.Get("test_fs_put_by_path.txt", fd)
iferr ! =nil {
	return err
}
defer fd.close(a)Copy the code

Gets an object to the specified object

GetToPath(key string, dest string) error

The GetToPath method writes from one object to another. This method checks permissions and object names on both paths, respectively. Dest address must be a relative path or an absolute path. Dest does not spell base dir.

if err := fsStorage.GetToPath("test_fs_put_by_path.txt".".. /test.txt") ; err != nil {
	return err
}
Copy the code

Obtaining Object Information

Stat(key string) (*FileInfo, error)

type FileInfo struct {
	Size    int64         / / size
	ModTime time.Time     // Change the time
	Mode    os.FileMode   / / permission
}
Copy the code

stat,err := fsStorage.Stat("test_get_to_path.txt"); 
iferr ! =nil {
	return err
}
fmt.Sprintf("%v\n",stat)
Copy the code

Delete the object

Del(key string) error

The Del method also returns an error if it deletes a file that does not have permissions or does not exist

if err := fsStorage.Del("test_get_to_path.txt"); err ! =nil {
	return err
}
Copy the code

Get object size

Size(key string) (int64, error)

It checks whether the object exists and returns an error if it does not

size,err := fsStorage.Size("test_get_to_path.txt"); 
iferr ! =nil {
	return err
}
fmt.Println(size)
Copy the code

Checks whether the object exists

IsExist(key string) (bool, error)

exist, err := fsStorage.IsExist("test.txt")
iferr ! =nil {
	return err
}
fmt.Sprintf("%v\n",exist)
Copy the code

RoadMap

Change the string format of config to match the configuration file directly, which is more convenient.

Provide bindings for primary and secondary stores, allowing multiple stores to be used within a project;

Others

Storage based on friend YMC on the basis of encapsulation again modified