Rsync, SCP, wget, curl, nc, FTP, LRZSZ transfer files

Article source: Tao Teacher operation notes – wechat official account

In this paper, the rsync, SCP, wget, curl, nc, FTP, LRZSZ tool use, especially in the file transfer applications. Main content from www.ibm.com/developerwo… Show appreciation.

1. rsync

Rsync is an efficient remote data backup and mirroring tool that can quickly synchronize files between multiple hosts. Liverpoolfc.tv: rsync.samba.org/

Setup of the Rsync service: Setup of the FTP and Rsync services for CentOS7

1.1 Basic Syntax

$man rsync
rsync [OPTION]... SRC DEST
rsync [OPTION]... SRC [USER@][host]:DEST
rsync [OPTION]... [USER@]HOST:SRC DEST
rsync [OPTION]... [USER@]HOST::SRC DEST
rsync [OPTION]... SRC [USER@]HOST::DEST
rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]

$rsync-avzp [email protected]::data1 ~/data1 Retain the original file transfer permission, user, user group, time, recursively copy data1 module data to the data1 directory in the user's home directory. Parameter A means to retain the original file permissions, users, user groups, time and recursive copy including linked files.$rsync --compress --recursive --times --perms --owner --group --links  --exclude=smarty/templates_c/  --timeout=30 [path] [user]@[ip]:[path]
Copy the code

Parameter Description:

options describe
-a, –archive Archive mode, keeping all file attributes, equivalent to -rlptGod
-v, –verbose Detail Output
-r, –recursive Recursive processing of subdirectories
-R, –relative Use relative path information
-b, –backup Create backup
-z, –compress Compress backup files during transfer
–delete Delete files that do not exist in SRC from DEST to synchronize directories
–progress Show the backup process

1.2 Command Examples

# set password:
$export RSYNC_PASSWORD=ops123456;
# the speed limit
$rsync macdown071.zip [email protected]::software/   --bwlimit=50000 -arv 
$rsync  --password-file=/etc/rsyncd.secrets.passfile ops@ip::software . --bwlimit=50000 -arv

# backup
$rsync-avzp [email protected]::data1 ~/data1 Retain the original file transfer permission, user, user group, time, recursively copy data1 module data to the user's home data1 directory.Delete - delete #Rsync-avzp --delete [email protected]::data1 ~/data1# keep data1 conformity module and user's home directory module, combined with parameter -- delete, it will delete in the ~ / data1 directory but not in data1
Copy the code

2. SCP

The SCP command copies files between hosts on the network. It stands for Secure Copy. The SCP command uses SSH to transfer data and uses the same authentication mode as SSH to provide the same security guarantee.

2.1 Basic Syntax

$man scp
$scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file] 
         [-l limit] [-o ssh_option] [-P port] [-S program] [[user@]host1:]file1 [...]  [[user@]host2:]file2Copy the code

2.2 Command Examples

# sample:
$scp [email protected]:/home/ops/foo.txt  ~/Downloads/
$scp-r -p 4588 [email protected]:/ TMP /foo. TXT /root/ Copy Local file /etc/eva.log to the home directory of user sysB$scpLog user@sysB:/home/user copy File /home/uesr/eva.log on the remote sysB to the /etc directory on the local PC and keep the file properties unchanged$scp-p user@sysB:/home/uesr/eva.log /etc copy sysB directory /home/user to /home/user/tmp, <new dir,/home/user/tmp/user>$scp -r user@sysB:/home/user /home/user/tmp
Copy the code

3. wget

Wget is a gPL-licensed free software package that automatically obtains files from the network. It is a non-interactive command line tool. Supports HTTP, HTTPS, and FTP, proxy servers, and resumable breakpoints. Wget enables recursive downloads, completely rebuilds the directory structure of the original site, and mirrors the remote site. During recursive downloads, WGET converts hyperlinks in the page to pointing to local files for offline browsing.

3.1 Basic Syntax

#http://www.gnu.org/software/wget/manual/wget.html

$man wget
wget [options] [URL-list]

Copy the code

Wget Help Manual: www.gnu.org/software/wg… Parameter Description:

options describe
-r Recursively download all directories and files on the server. The recursion depth is specified with the -l option.
-b The background to download
-m Creating a Site Image
-c Specifies the breakpoint continuation function. This feature requires that the server support resumable breakpoints.
-I Batch download can be realized by specifying a list of download directories
-A/-R Specify accept/reject download list for selective download
–proxy=on/off Specifies whether to use a proxy server for downloading
-t, –tries=NUMBER Maximum number of link attempts (0 means unlimited, default is 20)
-nc, –no-clobber Does not overwrite existing files
-N, –timestamping Only download new files than local
-nd –no-directories Do not create a directory structure
-x, –force-directories Force the creation of a directory structure
-nH, –no-host-directories The host directory structure is not inherited
-P, –directory-prefix=PREFIX Setting a directory Prefix

3.2 Command Examples

  • –http-user=’ XXX ‘–http-passwd=’ XXX’
# Download file
$wget https://dldir1.qq.com/weixin/mac/WeChatMac.dmg   -O WeChatMac.dmg
# have a password
$wget -P /data1 -c --http-user='xxx' --http-passwd='xxx' http://ip:80/scripts/install.sh

Copy the code
  • Downloading FTP Files
# Download FTP file
$wget -P /tmp/ ftp://ftp.tao-ops.com:21/pub/README.md --ftp-user=ftpuser --ftp-password=ftppwd
Copy the code
  • Download site
Download all the files needed to display a complete web page, such as images, etc.
# Do not search the upper directory in the download and convert absolute links to relative links.
$wget -r -p -np -k http://www.ibm.com.cn

The recursion depth is 4, and the number of retries is infinite
$wget -m -l4 -t0 http://www.ibm.com.cn
# Use proxy to download, and implement breakpoint continuation.
$wget -Y on -c http://www.ibm.com.cn
Copy the code

4. curl

Curl, which is a command-line tool wrapper for the libcurl library. Curl uses the URL syntax to transfer files. It supports FTP, FTPS, HTTP, HTTPS, TFTP, SFTP, and TELNET.

4.1 Basic Syntax

$man curl
curl [options] [URL...]
Copy the code

4.2 Command Examples

  • Curl prints the content from the specified URL to standard output. You can use -o if you want to save to a local file, or use the -o argument to specify the name of the file to use on the remote host.
curl http://www.baidu.com
curl -o 1.html http://www.baidu.com
curl -O http://www.baidu.com/index.html
Copy the code
  • If you need to specify the user name and password for downloading FTP files, you can specify the user name and password in the URL or use the -u parameter.
# password-less
$curl ftp://ftp.gnu.org/README
# have a password
$curl ftp://name:[email protected]:port/full/path/to/file -O 
$curl -u name:passwd ftp://machine.domain:port/full/path/to/file -o xxx.file
Copy the code
  • Using a proxy Use the -x option to connect using a proxy
curl -x my-proxy:port ftp://ftp.somesite.com/README
Copy the code

Curl is useful for automated file transfers or sequences of operations, and it’s a good tool for simulating user behavior in a Web browser. Libcurl is a good choice, especially if you need to call it from within your program.

5. nc

Nc named Netcat, can connect and redirect Sockets. The NC command has many functions. It can open TCP connections, send UDP packets, listen on TCP/UDP ports, scan ports, and so on. Nc is recommended for fast file transfer between two machines.

5.1 Basic Syntax

$man nc
ncat - Concatenate and redirect sockets
# usage
$ncat[OPTIONS...] [hostname] [port] [- nc hlnruz] [- g < > gateway...] [- g < number > pointer] [-i < seconds delay >] [-o < > output file] [-p < communication port >] [-s< source address >][-v...] [-w< timeout seconds >][host name][Communication port...]Copy the code

The basic parameters

options describe
4 – Use only the IPv4
– 6 Use only the IPv6
-c,–sh-exec [command] Run the given command, as in /bin/sh
-e,–exec [command] Execute the given command
-m,–max-conns [n] Sets the maximum number of simultaneous connections
-d,–delay [time] Set the read/write wait time
-o,–output [filename] Save the session data as a file
-i,–idle-timeout [time] Read/write idle timeout
-p,–source-port port Specifies the communication port used by this machine
-s,–source addr Specifies the host address for the remote communication
-l,–listen Listen for incoming connection information
-k,–keep-open Receive multiple connection information in listening mode
-n,–nodns Use the IP address instead of resolving the domain name
-t,–telnet Responding to a Telnet request
-u,–udp Replace the default TCP with UDP
-v,–verbose The detailed output information is displayed
-w,–wait [time] Set the connection timeout period, in seconds
-z Immediately after the connection is established, no data is sent or received, that is, zero-I /O mode, only used to test the connection status

5.2 Command Examples

  • Transferring files 1) Transferring files
Target machine (receiving machine): NC-l 1234 >mysql-bin.000001 
Source machine: the sender of the file, start the listening port and prepare the corresponding fileNc receiver IP 1234 < mysql-bin.000001Copy the code

2) Transfer folders

Target (receiving machine): 192.168.0.3NC-l1234 | tar XZVF - > mysql3306 source machine (sending machine) : the tar CZVF - mysql3306 | nc 192.168.0.3 1234Copy the code
  • Test port
Scan local ports 20-22$nc-z -v -n -w 2 127.0.0.1 20-22 NC: connect to 127.0.0.1 port 20 (TCP) failed: Connection refused nc: Connect to 127.0.0.1 port 21 (TCP) failed: Connection refused Connection to 127.0.0.1 22 port [TCP /*] Succeeded!#mysql
$nc -z -vv 192.110.103.41  3306
Connection to 192.110.103.41 3306 port [tcp/mysql] succeeded!
#zk
$echo 'ruok' | nc 127.0.0.1 2181
#memcache
$nc -z -vv 127.0.0.1 11211
Connection to 127.0.0.1 11211 port [tcp/memcache] succeeded!
Copy the code
  • Operating the memcached
yum install memcached
# start
memcached -d -p 11211 -m 256
#set key flags exptime bytes [noreply] 
$nc-z -vv 127.0.0.1 11211printf "set mykey 0 1000 6\r\n123456\r\n"| nc 127.0.0.1 11211 2) to get the data:printf "get mykey\r\n"| nc 127.0.0.1 11211 3) delete the data:printf "delete mykey\r\n"| nc 127.0.0.1 11211 4) to check the status:printf "stats\r\n"| nc 127.0.0.1 11211 5) to simulate the top command to check the status: watch"echo stats"| nc 127.0.0.1 11211 6) clear the cache:printf "flush_all\r\n"| nc 127.0.0.1 11211 (be careful operation, clear the cache will be gone)Copy the code
  • Chat Chat tool
On 192.168.2.34: nc-l1234 on 192.168.2.33: nc 192.168.2.34 1234 This way, the two parties can communicate with each other. Press CTRL +C(or D) to exit.Copy the code

6. ftp

The FTP command uses File Transfer Protocol (FTP) to Transfer files between a local host and a remote host or between two remote hosts. The FTP protocol allows data to be transferred between hosts with different file systems.

6.1 Basic Syntax

$FTP Host name /IP addressIf no hostname is specified, FTP will prompt the user to enter the command
# FTP command descriptionLs Lists the current directory of the remote machinecdChange the working directory on the remote computer LCD Change the working directory on the local computer ASCII Set the file transfer mode to ASCII mode binary Set the file transfer mode to binary mode close End the current FTP session GET (MGET) Transfers a specified file from the remote computer to the local computer Put (mput) Transfers specified files from the local machine to the remote machine. Open Connect to the remote FTP site. Quit Disconnect from the remote machine and exit the FTP? Display local help information! Go to the Shell prompt 1 to turn off the interactive modeCopy the code

6.2 Command Examples

# FTP get (mGET) Transfers specified files from the remote machine to the local machine
#wget 
$wget -P /tmp/ ftp://ftp.tao-ops.com:21/pub/README.md --ftp-user=user --ftp-password=ftppwd

Password # curl
$curl ftp://name:[email protected]:port/full/path/to/file -O 
$curl -u name:passwd ftp://machine.domain:port/full/path/to/file -o xxx.file
Copy the code

7. lrzsz

LRZSZ is a Unix communication suite provided by X, Y, and ZModem file transfer protocol, can be used between Windows and Linux system file transfer, small size and high speed.

7.1 Basic Syntax

# installation
$yum install lrzsz
$man rz
$man sz
Copy the code

7.2 Command Examples

If SecureCRT is used, open SecureCRT and choose Options > Session Options > X/Y/Zmodem to set upload and download directories.

# Download (upload files from client to Linux server)
$rz 
# upload (send file to client)
$sz /etc/passwd
Copy the code

8. Summary of transmission tools

File transfer method has its typical application scenarios based on its own characteristics and advantages:

  • Rsync is more suitable for daily synchronization of large amounts of data. The copy speed is very fast, which is faster, safer and more efficient than WGET.
  • FTP, as the most commonly used entry-level file transfer method, is simple to use, easy to understand, and can realize script automation;
  • SCP uses SSH to transmit data and uses the same authentication mode as SSH to provide stronger security than RCP.
  • Wget, achieve recursive download, can follow the HTML page on the link download in turn to create a local version of the remote server, completely rebuild the original site directory structure, suitable for the remote site mirror;
  • Curl, on the other hand, is useful for automated file transfers or operations, and is a good tool for simulating user behavior in a browser.
  • Nc The NC function enables you to quickly and conveniently transfer files or folders between two machines.
  • LRZSZ is suitable for short text transfer on Linux server and client.

Reference:

  • CentOS7 FTP and Rsync services are configured
  • www.ibm.com/developerwo…
  • memcached.org/
  • www.cnblogs.com/misswangxin…
  • Blog.csdn.net/jinxingfeng…