FastDFS overview

  • FastDFS is an open source lightweight distributed file system. It manages files, including file storage, file synchronization, and file access (file upload and download). It solves the problems of large-capacity storage and load balancing. It is especially suitable for online services with file as the carrier, such as photo album website, video website and so on.
  • FastDFS is tailor-made for the Internet. It takes into account redundant backup, load balancing, and linear expansion, and emphasizes high availability and performance. It is easy to set up a high-performance file server cluster to provide file uploading and downloading services.
  • FastDFS was developed by Yu Qing, a senior architect at Alibaba.

Fastdfs principle

FastDFS includes Tracker Server and Storage Server. The client requests the Tracker Server to upload and download files. The Tracker Server schedules the Storage Server to complete the upload and download.

  • A Tracker.

    • Function is load balancing and scheduling, it manages the Storage Server, can be understood as: “butler, tracker, dispatcher”;
    • The Tracker Server can be clustered to implement high availability and the policy is polling.

  • Storage

    • Files uploaded by the client are stored on the storage server.
    • A storage cluster is grouped. Each server in a storage group has an equal relationship and synchronizes data to back up data for high availability. Servers in different groups do not communicate with each other.
    • If the storage capacity of each server in the same group is inconsistent, the server with the smallest capacity is selected. Therefore, the software and hardware of the servers in the same group must be consistent.
    • The Storage Server connects to all the Tracker servers in the cluster and periodically reports its status to them, such as the remaining space, file synchronization status, and file upload and download times.

File Uploading ProcessFile Query Process

Install Fastdfs

1. Install GCC

yum install -y gcc gcc-c++
Copy the code

2. Download libfastcommon to /usr/local

CD/usr/local wget HTTP: / / https://github.com/happyfish100/libfastcommon/archive/V1.0.7.tar.gzCopy the code

3. Unzip libfastCommon

Tar -zxvf v1.0.7.tar. gz CD libfastcommon-1.0.7Copy the code

4. Install libfastCommon

./make.sh
./make.sh install
Copy the code

5. Download Fastdfs

Wget HTTP: / / https://github.com/happyfish100/fastdfs/archive/V5.05.tar.gzCopy the code

6. Unzip Fastdfs and install it

Tar -zxvf v5.05.tar. gz CD fastdfS-5.05 /./makeCopy the code

7. Copy all the files in the conf directory to /etc/fdf/

Cp/usr/local/fastdfs - 5.05 / conf / * / etc/FDFS /Copy the code

Configure the tracker

cd /etc/fdfs
vi tracker.conf
Copy the code

Main configuration

/usr/local/fastdfs base_path=/usr/local/fastdfsCopy the code

If base_path does not exist, you need to create a directory

mkdir /usr/local/fastdfs
Copy the code

9. Configure storage

cd /etc/fdfs
vi storage.conf
Copy the code

Main configuration

Group_name =group1 # port=23000 # heartbeat interval to tracker (s) heart_beat_interval=30 Set base_path=/usr/local/fastdfs; Store_path0 =/usr/local/fastdfs/fdfs_storage # tracker_server=192.168.31.168:22122Copy the code

Start the tracker

/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
Copy the code

Start the storage

/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart
Copy the code

Check the service

netstat -ntlp
Copy the code

Integrate the Nginx module

Upload the fastdfs-nginx-module_v1.16.tar.gz file to /usr/local

The tar - ZXVF fastdfs - nginx - module_v1. 16. The tar. GzCopy the code

3. Modify the config file to change the /usr/local/path to /usr/

cd /usr/local/fastdfs-nginx-module/src
vi config
Copy the code

4. Copy mod_fastdfs.conf from fastdfs-nginx-module/ SRC to /etc/fdfs

cp mod_fastdfs.conf /etc/fdfs/
Copy the code

5. Modify /etc/ff/mod_fastdfs.conf

Vi /etc/ff/mod_fastdfs. conf Contents: Base_path =/usr/local/fastdfs tracker_server=192.168.31.168:22122 # Url_have_group_name =true Store_path0 =/usr/local/fastdfs/fdfs_storageCopy the code

6. Copy libfdfsclient.so to /usr/lib

cp /usr/lib64/libfdfsclient.so /usr/lib/
Copy the code

Create nginx/client directory

mkdir -p /var/temp/nginx/client
Copy the code

Install Nginx

Run the following command to decompress the nginx-1.8.0.tar.gz file to /usr/local: tar -zxvf nginx-1.8.0.tar

yum install pcre
yum install pcre-devel
yum install zlib
yum install zlib-devel
yum install openssl
yum install openssl-devel
Copy the code

4, Go to the directory where nginx decompressed:

CD/usr/local/nginx - 1.8.0 comes withCopy the code

5, installation,

./configure \ --prefix=/usr/local/nginx \ --pid-path=/var/run/nginx/nginx.pid \ --lock-path=/var/lock/nginx.lock \ --error-log-path=/var/log/nginx/error.log \ --http-log-path=/var/log/nginx/access.log \ --with-http_gzip_static_module \  --http-client-body-temp-path=/var/temp/nginx/client \ --http-proxy-temp-path=/var/temp/nginx/proxy \ --http-fastcgi-temp-path=/var/temp/nginx/fastcgi \ --http-uwsgi-temp-path=/var/temp/nginx/uwsgi \ --http-scgi-temp-path=/var/temp/nginx/scgi \ --add-module=/usr/local/fastdfs-nginx-module/srcCopy the code

Compile and install

make
make install
Copy the code

6. Copy configuration files http.conf and mime.types

CD /usr/local/fastdfs-5.0.5/conf cp http.conf mime.types /etc/fdfs.Copy the code

7. Modify the nginx configuration file

cd /usr/local/nginx/conf/
vi nginx.conf
Copy the code

Close nginx and start nginx

pkill -9 nginx
/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
Copy the code

9. Start nginx

Integrate SpringBoot Fastdfs

Create a SpringBoot project. Import dependencies

<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-thymeleaf</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>com.github.tobato</groupId> <artifactId>fastdfs-client</artifactId> The < version > 1.26.4 < / version > < / dependency >Copy the code

3. Start the configuration on the class

@Import(FdfsClientConfig.class)
@EnableMBeanExport(registration = RegistrationPolicy.IGNORE_EXISTING)
Copy the code

4. Configuration files

fdfs.so-timeout=3000 fdfs.connect-timeout=1000 fdfs.thumb-image.height=60 fdfs.thumb-image.width=60 FDFS. Tracker - list = 192.168.31.168:22122Copy the code

5. Controller

@controller public class UploadController {public static final String DIR = "http://192.168.31.168/"; @Autowired private FastFileStorageClient client; @RequestMapping("login") public String login(){ return "login"; } @ResponseBody @RequestMapping(value = "/upload",method = RequestMethod.POST) public JsonResult upload(MultipartFile File) throws IOException {/ / get the suffix String extension = FilenameUtils. GetExtension (file. GetOriginalFilename ()); StorePath = client.uploadfile (file.getinputStream (), file.getSize(), extension, null); System.out.println("save:" + storePath.getFullPath()); return new JsonResult(1,DIR + storePath.getFullPath()); }}Copy the code

Java object

public class JsonResult {

    private Integer code;
    private Object data;
    //get/set/constructor
 }
Copy the code

The test page uses Vue+ElementUI

<! DOCTYPE HTML > < HTML lang="en" XMLNS :th="http://www.thymeleaf.org"> <head> <meta charset="UTF-8"> <title> Upload </title> <link  rel="stylesheet" href="/elementui/index.css"> <style> ... </style> </head> <body> <div id="app"> <el-card > <el-upload class="avatar-uploader" action="/upload" :show-file-list="false" :on-success="handleAvatarSuccess"> <img v-if="imageUrl" :src="imageUrl" class="avatar"> <i v-else class="el-icon-plus avatar-uploader-icon"></i> </el-upload> </el-card> </div> <script src="/vue/vue.js"></script>  <script src="/elementui/index.js"></script> <script> new Vue({ el:"#app", data(){ return{ imageUrl: '' } }, methods:{ handleAvatarSuccess(res, file) { console.log(res); this.imageUrl = res.data; } } }) </script> </body> </html>Copy the code

Upload the effect