Next-22 - Add Photo Albums series -2- Sync image directories with QShell

Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”.

This article describes how to use qshell to synchronize local file directories to storage space with a hierarchical structure.

The preparatory work

The need for a map bed to manage image folders
Apply for a Qiniu Cloud account
Open hexO-9 – Add Hexo-9

The environment

Directory of resources to upload:

$ tree1. ├ ─ ─ │ ├ ─ ─ a. pg │ ├ ─ ─ b.j pg │ └ ─ ─ SAN Antonio pg ├ ─ ─ 2 │ ├ ─ surprised ─ jeter pg │ ├ ─ ─ e.j pg │ └ ─ ─ f.j pg ├ ─ ─ g.j pg ├ ─ ─ h.j pg ├ ─ ─ i.j pg └ ─ ─ j.jpgCopy the code

Download tool

Command line tool qshell portal

The documentation required for the various operating systems is clear. I downloaded Qshell-linux-x64-v2.4.1.zip on Ubuntu18.04 x64

Extract qshell-linux-x64-v2.4.1

Rename it to qshell and give it executable permissions

mv qshell-linux-x64 qshell
chmod +x qshell
Copy the code

Configuring environment Variables

Add the qshell file location to ~/.bashrc

export PATH="/path of qshell:$PATH"
Copy the code

Bashrc makes the path take effect immediately

source ~/.bashrc 
Copy the code

If the version information is displayed, the configuration is successful

qshell -v

#Qshell version v2.4.1
Copy the code

Configure account

The commands to be authenticated depend on AccessKey and SecretKey of the seven cows account, located in the personal center -> SecretKey management

Account configuration:

qshell account [<AccessKey> <SecretKey> <Name>]
Copy the code

Then run the qshell account command again to verify that the setting is successful

qshell account

#Name: zywvvd
#The AccessKey: Bd0UYk1e9pFWK x x x x x x x x x x x x x x x
#SecretKey: ZFsdguzYE3QUl x x x x x x x x x x x x x x x
Copy the code

Generate the qshell directory in the current user’s home directory after doing this:

$ ls ~/.qshell/
# account.json
Copy the code

Write synchronous directory files

Create a configuration file in the ~/.qshell directory mentioned above, with the following contents:

$ cat upload.conf 
{
    "src_dir" : "/home/vvd/VVD_Work/test",
    "bucket" : "vvd-space"
}
Copy the code

Src_dir is the path where I just saved the photo

Bucket is your seven ox cloud space name

Sync files

Run the command in the ~/.qshell folder:

qshell qupload upload.conf
Copy the code

Return upload log:

Writing upload log to file /home/vvd/.qshell/qupload/040adc5d316ff8cfc274b2e099628234/040adc5d316ff8cfc274b2e099628234.log Uploading / home/VVD/VVD_Work/test / 1 / / a. a. pg = > 1 pg (1/10, 10.0%)... Uploading /home/ VVD /VVD_Work/test/1/ B.jpg => 1/ B.jpg [2/10, 20.0%]... Uploading /home/ VVD /VVD_Work/test/1/c. JPG => 1/ C. JPG [3/10, 30.0%]... Uploading /home/ VVD /VVD_Work/test/2/d. JPG => 2/ D. JPG [4/10, 40.0%]... Uploading /home/ VVD /VVD_Work/test/2/ E.jpg => 2/ E.jpg [5/10, 50.0%]... Uploading /home/ VVD /VVD_Work/test/2/ F.jpg => 2/ F.jpg [6/10, 60.0%]... Uploading /home/ VVD /VVD_Work/test/g.jpg => g.jpg [7/10, 70.0%]... Uploading /home/ VVD /VVD_Work/test/h.jpg => H.jpg [8/10, 80.0%]... Uploading /home/ VVD /VVD_Work/test/i.jpg => I.jpg [9/10, 90.0%]... Uploading /home/ VVD /VVD_Work/test/j.jpg => j.jpg [10/10, 100.0%]... See upload log at path /home/vvd/.qshell/qupload/040adc5dCopy the code

Viewing Synchronization Logs

The logs are stored in the qupload folder under ~/. Qshell:

Enter the seven Cloud console to view the uploaded files

In the console -> Space Management -> File Management you can see the synchronized files

Image file directory updated

When we update the original image directory (new folder, image new.jpg) :

Tree. ├ ─ ─ 1 │ ├ ─ ─ a. pg │ ├ ─ ─ b.j pg │ └ ─ ─ SAN Antonio pg ├ ─ ─ 2 │ ├ ─ surprised ─ jeter pg │ ├ ─ ─ e.j pg │ └ ─ ─ f.j pg ├ ─ ─ g.j pg ├ ─ ─ h.j pg ├ ─ ─ i.j pg ├── new ├─ newCopy the code

Methods a

Back to ~/.qshell upload file:

qshell qupload upload.conf Writing upload log to file /home/vvd/.qshell/qupload/c50b44d8de5ae468d8080f1f37fb946c/c50b44d8de5ae468d8080f1f37fb946c.log Uploading / home/VVD/VVD_Work/test / 1 / / a. a. pg = > 1 pg (1/8, 12.5%)... Uploading/home/VVD/VVD_Work/test / 1 / b.j pg = > 1 / b.j pg (2/8, 25.0%)... Uploading/home/VVD/VVD_Work/test / 1 / SAN Antonio pg = > 1 / SAN Antonio pg (3/8, 37.5%)... Uploading/home/VVD VVD_Work/test/surprised/surprised/jeter pg = > 2 / jeter pg (4/8, 50.0%)... Uploading/home/VVD/VVD_Work/test / 2 / e.j pg = > 2 / e.j pg (5/8, 62.5%)... Uploading/home/VVD/VVD_Work/test / 2 / f.j pg = > 2 / f.j pg (6/8, 75.0%)... Uploading /home/ VVD /VVD_Work/test/g.jpg => G.jpg [7/8, 87.5%]... Uploading /home/ VVD /VVD_Work/test/h.jpg => H.jpg [8/8, 100.0%]... See upload log at path /home/vvd/.qshell/qupload/c50b44d8de5ae468d8080f1f37fb946c/c50b44d8de5ae468d8080f1f37fb946c.logCopy the code

There is no update, I wonder if there are other functions to complete this function.

So I wrote a simple script upload.sh to implement real-time synchronization:

#! /bin/bash

rm  -rf  ~/.qshell/qupload
qshell  qupload  ~/.qshell/upload.conf
Copy the code

Use sh upload.sh instead of looking for the upload.conf file every time

Method 2

Cough cough, the somebody else seven niuyun has this setting, need not oneself laborious delete somebody else log.

Add configuration to upload.conf:

Rescan_local: local incremental upload is supported

Overwrite: local file overwrite upload is supported

{
    "src_dir" : "/home/vvd/VVD_Work/test",
    "bucket" : "vvd-space",
    "rescan_local" : true,
    "overwrite" : true
}
Copy the code

Just upload it again:

qshell qupload upload.conf Writing upload log to file /home/vvd/.qshell/qupload/c50b44d8de5ae468d8080f1f37fb946c/c50b44d8de5ae468d8080f1f37fb946c.log Uploading / home/VVD/VVD_Work/test / 1 / / a. a. pg = > 1 pg (1/12, 8.3%)... Uploading /home/ VVD /VVD_Work/test/1/ B.jpg => 1/ B.jpg [2/12, 16.7%]... Uploading /home/ VVD /VVD_Work/test/1/c.jpg => 1/ C.jpg [3/12, 25.0%]... Uploading /home/ VVD /VVD_Work/test/2/d. JPG => 2/ D. JPG [4/12, 33.3%]... Uploading /home/ VVD /VVD_Work/test/2/ E.jpg => 2/ E.jpg [5/12, 41.7%]... Uploading /home/ VVD /VVD_Work/test/2/ F.jpg => 2/ F.jpg [6/12, 50.0%]... Uploading /home/ VVD /VVD_Work/test/3/g.jpg => 3/g.jpg [7/12, 58.3%]... Uploading /home/ VVD /VVD_Work/test/g.jpg => G.jpg [8/12, 66.7%]... Uploading /home/ VVD /VVD_Work/test/h.jpg => H.jpg [9/12, 75.0%]... Uploading /home/ VVD /VVD_Work/test/i.jpg => I.jpg [10/12, 83.3%]... Uploading /home/ VVD /VVD_Work/test/j.jpg => j.jpg [11/12, 91.7%]... Uploading /home/ VVD /VVD_Work/test/new/new.jpg => new/new.jpg [12/12, 100.0%]... See upload log at path /home/vvd/.qshell/qupload/c50b44d8de5ae468d8080f1f37fb946c/c50b44d8de5ae468d8080f1f37fb946c.logCopy the code

Qshell qupload configuration

The Qupload function requires the configuration file. The parameters in the configuration file are as follows:

{
   "src_dir"            :   "<LocalPath>",
   "bucket"             :   "<Bucket>",
   "file_list"          :   "<FileList>",
   "key_prefix"         :   "<Key Prefix>",
   "up_host"            :   "<Upload Host>",
   "ignore_dir"         :   false,
   "overwrite"          :   false,
   "check_exists"       :   false,
   "check_hash"         :   false,
   "check_size"         :   false,
   "rescan_local"       :   true,
   "skip_file_prefixes" :   "test,demo,",
   "skip_path_prefixes" :   "hello/,temp/",
   "skip_fixed_strings" :   ".svn,.git",
   "skip_suffixes"      :   ".DS_Store,.exe",
   "log_file"           :   "upload.log",
   "log_level"          :   "info",
   "log_rotate"         :   1,
   "log_stdout"         :   false,
   "file_type"          :   0
}
Copy the code

Parameter names	describe	Optional parameters
src_dir	The local synchronization path is in full path format. The tool will synchronize all files under this directory. Directory soft connections in the local path are not supported. Be careful when using it under Windows`src_dir`The Settings follow`D:\\jemy\\backup`This way. That’s in the path`\`There should be two (`\ \`)	N
bucket	The name of the target space for synchronizing data, which can be public or private	N
file_list	List of files to be synchronized. The content of the file list must be relative to`src_dir`File relative path list, you can not specify, the tool will automatically obtain`src_dir`Below is a list of files. Please use the`dircache`Command to generate a list of files that you can manually remove unwanted lines	Y
up_host	The upload domain name is optional. Generally, this parameter is not required	Y
ignore_dir	Specifies whether to ignore the local path when saving files in the 7-ox space. The default value is False	Y
key_prefix	The prefix of the file name used when saving the file in seven cows space, defaults to an empty string	Y
overwrite	Whether to overwrite an existing file with the same name in the space.	Y
check_exists	Check whether files with the same name exist in the space before uploading each file. The default value is false	Y
check_hash	in`check_exists`Set to`true`Check whether the local file hash is the same as the spatial file hash. By default, no check is performed to save synchronization time	Y
check_size	in`check_exists`Set to`true`In the case of if`check_hash`for`false`, then you can set`check_size`for`true`Perform a simple size check to check whether the local file size is the same as the spatial file size	Y
skip_file_prefixes	Skip all files whose names (without relative paths) are prefixed with a string from this prefix list	Y
skip_path_prefixes	Skip all files whose paths (relative paths) are prefixed with strings in this prefix list	Y
skip_fixed_strings	Skip all files whose file paths (relative paths) contain strings from this string list	Y
skip_suffixes	Skip all files or directories with the suffix string in the suffix list	Y
rescan_local	By default, new files are not synchronized. You need to manually set this parameter to True to detect new files.	Y
log_level	Upload log output level. This parameter is optional`debug`.`info`.`warn`.`error`, the default`info`	Y
log_file	If the output file of the uploaded log is not specified, it will be output to the default file in the qshell working directory. The file name can be seen in terminal output	Y
log_rotate	The switchover period for uploading log files, expressed in days. The default value is 1 day	Y
log_stdout	The default value is false. You can set this parameter to true when debugging the upload function	Y
file_type	File storage type. The default value is`0`(Standard storage)`1`For low frequency storage	Y
delete_on_success	After the files are uploaded, local files are deleted to save disks. For example, in the log archive scenario, the default value is`false`To enable the function, set this parameter to`true`Can.	Y

Example configuration file

My configuration file uses common practical configurations for your reference:

{" src_DIR ": "/Hexo/Album_Temp", ### "bucket" : "vvd-space", ### resCAN_local" : True, ### support incremental upload (critical) "key_prefix": "VVd_hexo /", ### upload image prefix" check_exists": "Skip_suffixes ":true ## check whether files with the same name exist before uploading ".json,.log,.conf", ### overwrite: true ### can overwrite the original file}Copy the code

A link to an image after uploading:

Photos.zywvvd.com/vvd_hexo/20…

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Next-22 – Add Photo Albums series -2- Sync image directories with QShell

The preparatory work

The environment

Download tool

Configuring environment Variables

Configure account

Write synchronous directory files

Sync files

Viewing Synchronization Logs

Enter the seven Cloud console to view the uploaded files

Image file directory updated

Methods a

Method 2

Qshell qupload configuration

Example configuration file

Next-22 – Add Photo Albums series -2- Sync image directories with QShell

The preparatory work

The environment

Download tool

Configuring environment Variables

Configure account

Write synchronous directory files

Sync files

Viewing Synchronization Logs

Enter the seven Cloud console to view the uploaded files

Image file directory updated

Methods a

Method 2

Qshell qupload configuration

Example configuration file

Related Posts

next.js CSR

GitLab CI/CD front-end project

TypeScript will become massively popular