Technical Application Scenarios
Bilibili website has a large number of excellent video courses, and provides users with free download, too many files how to improve storage efficiency? How to ensure the download speed with a large number of users? Distributed file system. We can configure multiple servers according to the number of users in the region and connect them through the network to provide services for all users.
What file system
The file system manages and stores files. The operating system accesses files through interfaces provided by the file system.
Common file systems include FAT16/FAT32, NTFS, HFS, UFS, APFS, XFS, and Ext4.
What is a distributed file system
Physical storage resources managed by file systems are not directly connected to local nodes, but are connected to nodes through networks.
Its benefits:
- Expand the file system processing capacity of one computer to multiple computers simultaneously.
- One computer is down, and another provides data.
- Each computer can be placed in a different area, so that users can access nearby, increasing access speed.
Mainstream distributed file systems
NFS Network file system
GFS(googleFs)
GFS is a scalable distributed file system for large, distributed applications that access large amounts of data. It runs on normal hardware and provides fault tolerance.
HDFS(Hadoop)
Namenode is responsible for recording block file information
Client the Client will want to ask Namenode to store the file, and then it will actually block the file to Datanodes
The HDFS adopts a master/slave structure. An HDFS cluster consists of one name node and several data nodes. Name nodes store metadata of data. A completed data file is divided into several blocks and stored in data nodes.
Distributed file service provider
Ali’s OSS, Ali Cloud high availability cloud storage service. Seven cattle cloud storage Baidu cloud storage