Rsync

A file transfer application based on SSH protocol.

It introduces the method of slice comparison transmission, so it is very efficient to transfer similar files. Simply put, its algorithm efficiently slices the source and destination codes, compares them in r lines, and sends only the inconsistent ones. Rsync automatically determines the data block size based on the file size, or you can manually specify the data block size using commands. The recommended size ranges from 500 to 1000 bytes.

For our application scenario, OSS file synchronization, files are not modifiable, are full transfer, so there is no need to worry about efficiency; Rsync is also a great way to continue breakpoints.

Sersync

Sersync was developed by Kingsoft engineers based on inotify.

Inotify is a module of the Linux kernel that monitors file system changes. It monitors directories and notifies user-space applications of changes. However, its records only include directory changes, and do not record which files or directories were changed.

Sersync can record changes (including additions, deletions, and modifications) of a specific file or directory name in the monitored directory, and it is a persistent record, so it can be compared with historical data. For example, start sersync, then stop sersync, modify files in the directory, and then start sersync. Sersync compares the current directory with the historical directory.

Sersync has the built-in rsync command to automatically synchronize files or directories that have changed.

Bidirectional synchronization of directories on different servers

Server A monitors the change of the target directory and sends the change to the rsync server of server B using the rsync client command.

Server B monitors the change of the target directory and sends the change to the rsync server of server A using the rsync client command.

The following has not been fully verified:

After A synchronizes changes to B, B checks the newly changed files and sends features to A. Server A finds that the signature codes are identical, which means that server B received the synchronization request. The chain is broken.