If you want to build an elegant, simple, full-featured static resource service, use MinIO.

1 the introduction

In the process of Web project development, we often need to deal with static resources (such as images, videos, audio, JS libraries, CSS libraries, etc.). Generally speaking, if the project needs to use these resource files, we commonly use the following methods:

  • Local storage: static in the project directory, establish js/CSS/icon/font/image/lib/audio/video directory, where the resources of the corresponding format files, respectively. When used, use relative location references in HTML files.
  • Use proxy to build static resource services: that is, store resources in a file directory and use proxy servers (such as Nginx and Apache) to map the directory to build resource services. When used, the url address of the proxy service is referenced in an HTML file.
  • Use third-party tools to set up static resource services: Use third-party open source file storage or object storage tools, or write your own program to implement the interface that can obtain files. The url address or interface address is used.
  • Use online static resource services, such as CDN.

For local storage, the disadvantages are obvious: resources are mixed with code files, unnecessary and not easy to expand. For locally deployed applications, it is obviously safer to build static resource services. For the use of proxy services and third-party tools, in contrast, proxy services only do mapping, although available, but a single function, only do mapping, no other management functions, also not easy to expand. Using third-party file or object storage tools is a good choice for file management, high scalability, high performance, and high availability. MinIO is such a user-friendly object storage tool. It is simple, fast, and fully functional.

This article is through the MinIO installation, configuration and use, build static resource services, so as to picture, video, audio, third-party JS library and other resources independent deployment, access; A brief introduction to the Java API provided by MinIO will also be provided for further development.

2 MinIO profile

According to the official introduction of MinIO, MinIO is a high-performance object storage (for details about the differences between block storage, file storage, and object storage, see distributed object storage solutions known to architects). It is compatible with Amazon S3 apis and fully considers the requirements and experience of developers. Supports distributed storage with high scalability and availability. Simple deployment but feature-rich. The official documentation is also detailed. There are many different deployment modes (standalone, distributed). MinIO is easy to use because it is easy to get up, run, and configure. You can use docker to install and run, or you can download binary files, and then use scripts to run.

This article explains this in the simplest way, on a Linux machine, on a single machine deployment, running binaries.

3 MinIO running and static resource usage

3.1 MinIO gets

MinIO development documentation, the download address is as follows:

  • linux: https://dl.min.io/server/minio/release/linux-amd64/minio
  • windows: https://dl.min.io/server/minio/release/windows-amd64/minio.exe

This article runs on Linux.

3.2 Starting and Running of MinIO

3.2.1 Simple start of the foreground

Save the downloaded minio file to a directory (for example, /opt/minio) as the running directory. Create a directory (for example, /opt/minio-data) as the miniO data storage directory to start the system. The script is as follows:

cd /opt/minio
chmod +x minio
./minio server /opt/minio-data
Copy the code

After the miniO is started, the system outputs the access address, access_key, and secret_key. Use the browser to access the endpoint address. If the access is successful, the miniO is installed. There are a few downsides to getting started:

  • Not running in the background,ctrl+cThe process ends and the service stops
  • No user name or password for custom access
  • No access IP or port is specified
  • Logs are not saved to files

To solve these problems, you are advised to use the following method to start the operation.

3.2.2 Background run with specified parameters

Run the program in the background with nohup, specifying password parameters, access address parameters, and log output, as shown below.

MINIO_ACCESS_KEY=minio MINIO_SECRET_KEY=minio123 nohup /opt/minio/minio  server --address "${MINIO_HOST}:${MINIO_PORT}" /opt/minio-data  > /opt/minio/minio.log 2>&1 &
Copy the code

MINIO_ACCESS_KEY and MINIO_SECRET_KEY are access passwords

${MINIO_HOST}:${MINIO_PORT} indicates the host and port to be accessed. Change the value as required.

${MINIO_HOST}:${MINIO_PORT} with the specified MINIO_ACCESS_KEY and MINIO_SECRET_KEY.

3.2.3 Creating a bucket and specifying an access policy

In the browser, log in to the MinIO storage system and click in the lower right corner to create buckets to store objects. Create the corresponding buckets for storing static resources: image, video, and Audio. In this way, the file can be uploaded in the corresponding bucket according to the resource type. After uploading, the file can be shared, and other places can obtain resources through the shared URL, as shown in the following figure.

The default MinIO policy is to share an address for a maximum of seven days. To break this limit, you can set the policy in a bucket. Click the corresponding bucket, edit Policy and add the policy *.*, Read Only, as follows:

This liberalizes access with no time limit and allows direct access to resources (no sharing required) by simply pressing http://${MINIO_HOST}:${MINIO_PORT}/${bucketName}/${fileName}.

Myths about MinIO directories

  • In fact, for object storage, there is no distinction between files and directories. All files and directories are objects, that is, image/temp/xxx.jpg and image/temp/ are objects. It is fundamentally different from the tree structure of the operating system file system.
  • When uploading a file, objectName can be/temp/xxx.jpg, the system automatically creates a temp directory.
  • MinIO does not provide operations like deleting a directory and deleting all files in that directory at the same time (i.erm -rf image/tempTo delete the image/temp directory, delete all files prefixed with image/temp.
  • When querying multiple files, you can use prefix matching, as described in the API documentationlistObjects(bucketName, prefix, recursive)

3.3 Referencing static Resources in HTML files

By setting up and running MinIO as a static resource server, you can write an HTML reference to static resources in MinIO. The following is the resource address of MinIO for the images, videos and audio in the TESTED HTML.

<div class="img-list">
        <img src="http://${MINIO_HOST}:${MINIO_PORT}/image/test.jpg" alt="Image">
    </div>
    <div class="audio-list">
        <audio src="http://${MINIO_HOST}:${MINIO_PORT}/audio/test.mp3"
        controls="controls"></audio>
    </div>
    <div class="video-list">
        <video src="http://${MINIO_HOST}:${MINIO_PORT}/video/test.mp4" controls="controls"></video>
    </div>
Copy the code

Discoverable resources are normally loaded and accessible.

4 Java client API operations

MinIO is very developer-friendly and provides apis in various languages. For details, refer to the MinIO development documentation. The following tests take Java as an example.

4.1 Adding a Dependency

<dependency>
    <groupId>io.minio</groupId>
    <artifactId>minio</artifactId>
    <version>6.0.13</version>
</dependency>
Copy the code

4.2 Operating files using the Java API

MinioClient = new minioClient (endpoint, accessKey, secretKey); , where the endpoint is the access address of MinIO, and the last two points correspond to the password set during startup.

4.2.1 Uploading files

/** * Upload file *@paramMinioClient Operates on the client *@paramBucketName Name of the bucket to be uploaded *@paramObjectName Name of the file stored in the bucket after the upload *@paramFilePath local filePath */
public void uploadFile(MinioClient minioClient, String bucketName, String objectName, String filePath) throws XmlPullParserException, NoSuchAlgorithmException, InvalidKeyException, IOException {
    try {
        // If no bucket exists, create a new one
        boolean isExist = minioClient.bucketExists(bucketName);
        if(! isExist) { minioClient.makeBucket(bucketName); }// Upload files using putObject
        minioClient.putObject(bucketName, objectName, filePath, null.null.null.null);
    } catch (MinioException e) {
        System.out.println("Error occurred: "+ e); }}Copy the code

4.2.2 Downloading files

/** * Download file **@paramMinioClient Operates on the client *@paramBucketName Name of the bucket to be uploaded *@paramObjectName Name of the file stored in the bucket after the upload *@paramDownloadPath Path for saving the downloaded file */
public void downloadFile(MinioClient minioClient, String bucketName, String objectName, String downloadPath) throws XmlPullParserException, NoSuchAlgorithmException, InvalidKeyException, IOException {
    File file = new File(downloadPath);
    try (OutputStream out = new FileOutputStream(file)) {
        InputStream inputStream = minioClient.getObject(bucketName, objectName);
        byte[] tempbytes = new byte[1024];
        int byteread = 0;
        while((byteread = inputStream.read(tempbytes)) ! = -1) {
            out.write(tempbytes, 0, byteread); }}catch (MinioException e) {
        System.out.println("Error occurred: "+ e); }}Copy the code

4.2.3 Deleting files

To delete a file, use removeObject.

minioClient.removeObject(bucketName, objectName);
Copy the code

4.2.4 Listing files

/** * list files *@param minioClient
  * @param bucketName
  */
public void listFile(MinioClient minioClient, String bucketName) throws XmlPullParserException, NoSuchAlgorithmException, InvalidKeyException, IOException {
    try {
        Iterable<Result<Item>> results = minioClient.listObjects(bucketName);
        Iterator<Result<Item>> iterator = results.iterator();
        while (iterator.hasNext()) {
            Item item = iterator.next().get();
            System.out.println(item.objectName() + "," + item.objectSize() + "B"); }}catch (MinioException e) {
        System.out.println("Error occurred: "+ e); }}Copy the code

5 concludes

Because of the need for independent access to static resources, dynamic and static separation, through the use of MinIO, you can quickly and easily achieve static resource server for access. This article describes how to download, deploy, start, run, and configure MinIO, and uses HTML to reference static resource files to explain how to use MinIO, and provides a simple use of Java API. I hope it’s helpful.

Download resources

In this article, nohup is used to start MinIO, but the command is too long, we usually write a script to start, close and query the status, so I write the script as a complete sh file for you to use. In addition, the testing of the MinIO Java API, which this example uses as a Spring Boot project, is carried out as a unit test.

The SH script file is included with the Spring Boot project in my Minio Github sample (Script Minio-Serviced. Sh is available for download if required.

Script usage:

  • Modify parameters in the sh script as required
  • Modified Executive permission: CHmod + X miniO-Serviced. Sh serviced
  • According to the parameters of the startup/shutdown/restart/running state:. / minio – serviced. Sh start/stop/restart/status

The resources

  • MinIO website: https://min.io/
  • MinIO development documentation: https://docs.min.io/
  • Practice of object storage scheme based on Go open source project MIMIO:https://mp.weixin.qq.com/s/MzA4ODg0NDkzOA==&mid=2247487119&idx=1&sn=6e09abb32392e015911be3a1d7f066e5&source=41
  • Distributed object storage solutions known to architects:https://juejin.cn/post/6844903844648845319
  • Build high-performance object storage using MiniO:https://tonybai.com/2020/03/16/build-high-performance-object-storage-with-minio-part1-prototype

The articles

  • SpringBoot Multiple data sources (3) : Parameterize change sources
  • SpringBoot Multiple Data Sources (2) : dynamic data sources
  • SpringBoot Multiple Data Sources (1) : Multiple source policies
  • Java development must learn: dynamic proxy
  • Good books to read in 2019
  • Springboot + Apache HTTPS is deployed on the front and back ends
  • Springboot + Logback Log Output
  • Springboot + Logback Log output

My official account (search Mason technical records) for more technical records: