Principle and deployment of FastDFS

FastDFS is an open source and lightweight distributed file system written in c language. It manages file processes, including file storage, file synchronization, file access (file upload, file download), etc. It solves the problem of mass storage and load balancing. It is especially suitable for online services based on files, such as photo album websites, video websites, etc.Tailored for the Internet, fully considering redundant backup, load balancing, current expansion, and focusing on high availability, high performance and other indicators, it is easy to set up a high-performance file server cluster using FastDFS to provide file upload and download services.

1. Detailed explanation of FastDFS principle and structure

1. FastDFS architecture
The FastDFS architecture includes Tracker server and Storage server. client requests Tracker server for file upload and download. File upload and download are finally completed by Storage server through Tracker server scheduling.

Tracker server: responsible for scheduling and load balancing. Through Tracker server, Storage server can be found to provide upload service when uploading files according to some strategies. tracker can be called a tracking server or a scheduling server.
Storage server: responsible for the final storage of files, the files uploaded by the client are ultimately stored on the storage server. Storage server does not implement its own file system, but uses the operating system's file system to manage files, which can be called a storage server.


2. Role of Fast DFS System
The Fast DFS system has three roles: Tracker Server, Storage Server, and Client.

Tracker Server: Track the server, mainly for scheduling and balancing purposes; manages all storage servers and groups, each of which connects to Tracker after startup, informs itself of the group it belongs to, and maintains a periodic heartbeat.
Storage Server: A storage server that provides capacity and backup services; in groups, there can be more than one storage server inside each group, with data backed up to each other.
Client: Client, the server that uploads and downloads data, the server where our own project is deployed.

3. Fast DFS Principle
About modules: Tracking Server and Storage Node can be composed of one or more servers, Tracking Server and Storage Node can increase or go offline at any time without affecting online services, where all servers in Tracking Server are equal and can increase or decrease at any time depending on server pressure

4. File upload process
Storage server s connect to all the Tracker server s in the cluster and report their status regularly, including statistics such as disk space left, file synchronization status, file uploads and downloads.


After the client uploads the file, the storage server returns the file ID to the client, which is used to access the index information of the file later.
File index information includes: group name, virtual disk path, data level directory, file name.

Group name: The name of the storage group in which the file is uploaded. The storage server returns after the file is uploaded successfully and the client needs to save it.
Virtual disk path: The virtual path of the storage configuration, corresponding to the disk option store_path*.If store_path0 is configured it is M00, if store_path1 is configured it is M01, and so on.
Two-level directories for data: A two-level directory created by the storage server under each virtual disk path to store data files.
File name: different from file upload.Generated by the storage server based on specific information, the file name contains information such as the source storage server IP address, file creation timestamp, file size, random number, file extension name, and so on.

5. File download process

client sends a download request to a tracker, which must take file name information with it. Tracker parses the group, size, creation time of the file from the file name, and then chooses a storage for the request to serve the read request. Since the files within the group are synchronized asynchronously in the background, it is possible that when reading, the files are not synchronized to some storage yet.On the server, to minimize access to such storage, the tracker chooses the readable storage within the group according to the following rules:
The time stamp to which the file creation timestamp-storage is synchronized and (current time-file creation timestamp)>Maximum file synchronization time (5 minutes), indicating that after the file creation, it is assumed that after the maximum synchronization time, it must have been synchronized to another storage.

II. Deployment of Fast DFS

1. The environment is as follows

All packages used in blog posts are available this (Extract Code: koxu) Download

Since there are some repetitive installation operations, find the big man to write a script that you can execute when you do those repetitive operations.There is this script in the link above
2. Configure tracker1

[root@tracker1 ~]# mkdir fastdfs
[root@tracker1 ~]# cd fastdfs/
[root@tracker1 fastdfs]# ls
fastdfs.tar.gz  install.sh  libfastcommon.tar.gz
[root@tracker1 fastdfs]# sh install.sh            # Execute script
tracker server install succees .......
[root@tracker1 fastdfs]# /etc/init.d/fdfs_trackerd start            # Start Service
Reloading systemd:                                         [  OK  ]
Starting fdfs_trackerd (via systemctl):                    [  OK  ]
[root@tracker1 fastdfs]# netstat -anput | grep 22122          # Confirm that the port is listening
tcp        0      0 192.168.171.134:22122   0.0.0.0:*               LISTEN      3847/fdfs_trackerd  

3. Configure tracker2

[root@tracker2 ~]# mkdir fastdfs
[root@tracker2 ~]# cd fastdfs/
[root@tracker2 fastdfs]# ls
fastdfs.tar.gz  install.sh  libfastcommon.tar.gz
[root@tracker2 fastdfs]# sh install.sh
tracker server install succees .......
[root@tracker2 fastdfs]# /etc/init.d/fdfs_trackerd start 
Reloading systemd:                                         [  OK  ]
Starting fdfs_trackerd (via systemctl):                    [  OK  ]
[root@tracker2 fastdfs]# netstat -anput | grep 22122
tcp        0      0 192.168.171.135:22122   0.0.0.0:*               LISTEN      4276/fdfs_trackerd  

4. Configure storage1

[root@storage1 ~]# mkdir fastdfs
[root@storage1 ~]# cd fastdfs/
[root@storage1 fastdfs]# tar zxf libfastcommon.tar.gz         
[root@storage1 fastdfs]# tar zxf fastdfs.tar.gz 
[root@storage1 libfastcommon-1.0.43]# ./make.sh && ./make.sh install
[root@storage1 libfastcommon-1.0.43]# cd ../fastdfs-6.06/
[root@storage1 fastdfs-6.06]# ./make.sh && ./make.sh install 
[root@storage1 fastdfs-6.06]# cd conf/
[root@storage1 conf]# cp mime.types http.conf /etc/fdfs/
[root@storage1 conf]# cd ../../
[root@storage1 fastdfs]# tar zxf fastdfs-nginx-module.tar.gz 
[root@storage1 fastdfs]# cp fastdfs-nginx-module-1.22/src/mod_fastdfs.conf /etc/fdfs/
[root@storage1 fastdfs]# cd /etc/fdfs/
[root@storage1 fdfs]# mkdir -p /data/storage-fdfs/base        # Suggest separate placement
[root@storage1 fdfs]# mkdir -p /data/storage-fdfs/store
[root@storage1 fdfs]# cp storage.conf.sample storage.conf
[root@storage1 fdfs]# vim storage.conf
group_name = group1
bind_addr =192.168.171.140
base_path = /data/storage-fdfs/base           #Data and Log Directory Address
store_path0 = /data/storage-fdfs/store             # Storage
tracker_server = 192.168.171.134:22122             #The following is the listening address for the specified tracker-server
tracker_server = 192.168.171.135:22122
http.server_port = 8888                 #Port for http access to files
[root@storage1 fdfs]# vim mod_fastdfs.conf 
base_path=/data/storage-fdfs/base
tracker_server=192.168.171.134:22122
tracker_server=192.168.171.135:22122
storage_server_port=23000
group_name=group1
url_have_group_name = true            #Change to true when group has more than one to access by group name
store_path0=/data/storage-fdfs/store
group_count = 2
[group1]
group_name=group1
storage_server_port=23000
store_path_count=1
store_path0=/data/storage-fdfs/store
[group2]
group_name=group2
storage_server_port=23000
store_path_count=1
store_path0=/data/storage-fdfs/store
#Compile and install nginx and add third-party modules
[root@storage1 fdfs]# cd ~/fastdfs/
[root@storage1 fastdfs]# yum -y install pcre-devel openssl-devel 
[root@storage1 fastdfs]# tar zxf nginx-1.16.1.tar.gz 
#Note the following third-party nginx module path, to correspond with your own
[root@storage1 fastdfs]# cd nginx-1.16.1/
[root@storage1 nginx-1.16.1]# ./configure --add-module=/root/fastdfs/fastdfs-nginx-module-1.22/src && make && make install 
[root@storage1 nginx-1.16.1]# vim /usr/local/nginx/conf/nginx.conf
#Add the following above the original server field (line 34)
    server {
        listen 8888;
        server_name localhost;
        location ~/group[0-9]/M00/ {
                ngx_fastdfs_module;
        }
}   
[root@storage1 nginx-1.16.1]# /etc/init.d/fdfs_storaged start 
Starting fdfs_storaged (via systemctl):                    [  Determine  ]
[root@storage1 nginx-1.16.1]# /usr/local/nginx/sbin/nginx 
ngx_http_fastdfs_set pid=16712
#Determine that the relevant port is listening
[root@storage01 conf]# netstat -anput | grep 23000
[root@storage01 conf]# netstat -anpt | grep 80
[root@storage01 conf]# netstat -anpt | grep 8888

4. Configure host storage02

[root@storage2 fastdfs]# tar zxf libfastcommon.tar.gz 
[root@storage2 fastdfs]# tar zxf fastdfs.tar.gz 
[root@storage2 fastdfs]# cd libfastcommon-1.0.43/
[root@storage2 libfastcommon-1.0.43]# ./make.sh && ./make.sh install 
[root@storage2 libfastcommon-1.0.43]# cd ../fastdfs-6.06/
[root@storage2 fastdfs-6.06]# ./make.sh && ./make.sh install 
[root@storage2 fastdfs-6.06]# cd conf/
[root@storage2 conf]# cp mime.types http.conf /etc/fdfs/
[root@storage2 conf]# cd ../../
[root@storage2 fastdfs]# tar zxf fastdfs-nginx-module.tar.gz 
[root@storage2 fastdfs]# cd /etc/fdfs/
#Copy the modified storage01 configuration file locally
[root@storage2 fdfs]# scp root@192.168.171.140:/etc/fdfs/storage.conf /etc/fdfs/
[root@storage2 fdfs]# scp root@192.168.171.140:/etc/fdfs/mod_fastdfs.conf /etc/fdfs/
[root@storage2 fdfs]# vim storage.conf           # Modify the copied configuration file
group_name = group2
bind_addr =192.168.171.143
[root@storage2 fdfs]# vim mod_fastdfs.conf 
group_name=group2
[root@storage2 fdfs]# mkdir -p /data/storage-fdfs/base
[root@storage2 fdfs]# mkdir -p /data/storage-fdfs/store
#Install nginx
[root@storage2 fdfs]# cd ~/fastdfs/
[root@storage2 fastdfs]# tar zxf nginx-1.16.1.tar.gz 
[root@storage2 fastdfs]# yum -y install pcre-devel openssl-devel 
[root@storage2 nginx-1.16.1]# ./configure --add-module=/root/fastdfs/fastdfs-nginx-module-1.22/src/ && make && make install 
[root@storage2 nginx-1.16.1]# cd /usr/local/nginx/conf/
#Copy the configuration file of the storage01 host
[root@storage2 conf]# scp root@192.168.171.140:/usr/local/nginx/conf/nginx.conf ./
[root@storage2 ~]# /etc/init.d/fdfs_storaged start 
Starting fdfs_storaged (via systemctl):                    [  Determine  ]
[root@storage2 ~]# /usr/local/nginx/sbin/nginx 
ngx_http_fastdfs_set pid=5694
[root@storage02 conf]# netstat -anput | grep 23000
[root@storage02 conf]# netstat -anpt | grep 80
[root@storage02 conf]# netstat -anpt | grep 8888

5. Configure host nginx to provide reverse proxy function

[root@nginx ~]# tar zxf nginx-1.16.1.tar.gz 
[root@nginx ~]# yum -y install pcre-devel openssl-devel 
[root@nginx ~]# cd nginx-1.16.1/
[root@nginx nginx-1.16.1]# ./configure && make && make install 
[root@nginx nginx-1.16.1]# vim /usr/local/nginx/conf/nginx.conf
#Find http field
http {
    include       mime.types;
    default_type  application/octet-stream;
    #Add the following
    upstream fdfs_group1 {
        server 192.168.171.140:8888 weight=1 max_fails=2 fail_timeout=30s;
        }
    upstream fdfs_group2 {
        server 192.168.171.143:8888 weight=1 max_fails=2 fail_timeout=30s;
        }
    #Find the server field
 server {
        listen       80;
        server_name  localhost
        #charset koi8-r
        #access_log  logs/host.access.log  main;
        location / {
            root   html;
            index  index.html index.htm;
        }
                #Add the following
        location ~/group1 {
                proxy_pass http://fdfs_group1;
        }
        location ~/group2 {
                proxy_pass http://fdfs_group2;
        }
[root@nginx nginx-1.16.1]# /usr/local/nginx/sbin/nginx 
[root@nginx nginx-1.16.1]# netstat -anput |grep 80

6. Client Validation Service Is OK

[root@client ~]# mkdir fastdfs
[root@client ~]# cd fastdfs/
[root@client fastdfs]# tar zxf libfastcommon.tar.gz 
[root@client fastdfs]# tar zxf fastdfs.tar.gz 
[root@client fastdfs]# cd libfastcommon-1.0.43/
[root@client libfastcommon-1.0.43]# ./make.sh && ./make.sh install 
[root@client libfastcommon-1.0.43]# cd ../fastdfs-6.06/
[root@client fastdfs-6.06]# ./make.sh && ./make.sh install 
[root@client fastdfs-6.06]# cd /etc/fdfs/
[root@client fdfs]# cp client.conf.sample client.conf
[root@client fdfs]# vim /etc/fdfs/client.conf
base_path = /data/storage-fdfs/base
tracker_server = 192.168.171.134:22122
tracker_server = 192.168.171.135:22122
[root@client ~]# mkdir -p /data/storage-fdfs/store
[root@client ~]# ls test.jpg 
test.jpg
[root@client ~]# fdfs_upload_file /etc/fdfs/client.conf test.jpg          # upload
group2/M00/00/00/wKirj15-NhGAIeapAAFEF6rSzeI524.jpg 
#The information returned above needs to be saved and accessed
[root@client ~]# fdfs_download_file /etc/fdfs/client.conf group2/M00/00/00/wKirj15-NhGAIeapAAFEF6rSzeI524.jpg b.jpg
[root@client ~]# ls b.jpg 
b.jpg                 # You can see that you have downloaded and changed the name

ID returned by browser when accessing nginx reverse proxy + uploading pictures

Last few FastDFS commands

fdfs_upload_file  Usage: 
fdfs_upload_file <config_file> <local_filename> [storage_ip:port] [store_path_index] 
For uploading files  
Usage is fdfs_upload_file +configuration file+file 
fdfs_download_file  Usage: 
fdfs_download_file <config_file> <file_id> [local_filename] [<download_offset> <download_bytes>] 
For downloading files  
Usage is fdfs_download_file +configuration file+file 
fdfs_file_info Usage: 
fdfs_file_info <config_file> <file_id> 
Used to view file information  
Usage is fdfs_file_info +profile+file 
fdfs_delete_file  Usage: 
fdfs_delete_file <config_file> <file_id> 
Used to delete files  
Usage is fdfs_delete_file +configuration file+file 
fdfs_monitor  Usage: 
fdfs_monitor <config_file> [-h <tracker_server>] [list|delete|set_trunk_server <group_name> [storage_id]] 
Used to view cluster information  
Usage is fdfs_monitor +profile 

Tags: Nginx vim yum OpenSSL

Posted on Sat, 28 Mar 2020 18:27:16 -0700 by tym