Welcome to Tencent Cloud + community, get more Tencent mass technology practice dry goods oh ~

This article was published by Xu Dengbo in cloud + Community column

Original statement: This article first cloud · cloud + community, without permission, shall not be reproduced

preface

The StorageNetworking Industry Association (SNIA) defines a snapshot as a fully usable copy of a specified data set. The copy contains the image of the corresponding data at a certain point in time (the time when the copy started). A snapshot can be a copy or a replica of the data it represents.

Note that a snapshot is a fully usable copy, but not a complete copy, as we’ll explain later.

Usage scenarios of snapshot storage

Scene 1:

Snapshot storage is a data protection measure that protects source data to a certain extent. Colloquially speaking, it can be understood as —- Regret medicine.

As shown in the figure above, assume that there is a complete source data at time T0, and we create a snapshot for this source data at time T1.

Time t2, if because of various reasons (wrong operation, system error, etc.) result in source data is damaged, so we can rollback (rollback) snapshot, will return to the source data snapshot creation state (i.e., time t1), in this way, you can try to reduce the data loss (loss of data, is data between the t1 and t2).

This function is often used in banks, public security household registration, scientific research units and so on. When the operating system or software is upgraded or equipment room is replaced, high-risk operations are performed at night or when no production services are running. Before the operations, snapshots are taken. If the operations fail, the snapshots are rolled back to restore the source data to the original state.

Scenario 2:

As mentioned in the introduction, a snapshot is a fully usable copy, so it can be used as the source data by the upper level business.

As shown in the preceding figure, after a snapshot is created for source data, the snapshot volume can be mapped to other upper-layer services for data mining and development testing. Read operations on the snapshot do not affect the data on the source volume.

This functionality, used to live (video & image) as a yellow, the scientific research data simulation development testing, etc., for example, live video platform needs to be a certain period of time of video provided to law enforcement agencies screening analysis, you can pass on a particular point in time saved data to create a snapshot, map the snapshot to law enforcement agencies, business host to mining is analyzed.

The implementation principle of snapshot storage

Currently, snapshot implementation methods are determined by vendors. However, there are two main technologies: Copy-on-write COW (COPy-on-write COW) and Write Redirect On Write (ROW).

Copy COW while writing

Copy-on-write (COW) refers to copy-on-write or copy-on-write.

If the data on the source volume changes after a snapshot is created, the snapshot system copies the original data to the corresponding data block on the snapshot volume before rewriting the source volume.

Write operations:

Pictured above a brief example, after snapshot creation, if the upper business writing data to the wave source X, X in the cache line, snapshot system will position (logical address) of X will be written to the data on the Y, copy to the corresponding locations in a snapshot volume (logical address), at the same time, generate a mapping table, a table column to record data on the source volume change of logical address, The other column records the logical address of the data change on the snapshot volume. For each data block delivered by the upper-layer service, two write operations occur: the source volume writes data to the snapshot volume (Y in the figure), and the upper-layer service writes data to the source volume (X in the figure).

Read:

Pictured above, if the snapshot volume mapping to analyze the data to the upper business purposes, the snapshot for read operations, the first by the snapshot system, whether the upper business needs to read data in the snapshot volume, if in, read from the snapshot volume directly, if not, the query mapping table, to logically reads the corresponding source volume (the look-up table and go to the source volume read operation, Also called read redirection). This explains why a snapshot is a fully usable copy. It does not make a 100% copy of the source volume, but for upper-layer services, a snapshot can be regarded as an “exact copy” of the source volume.

There is no data interaction between the source volume and the snapshot volume.

We can see that the snapshot to the source volume of data, has the very good protection measures snapshots can separate as a copy of the can read, but not like a mirror image of simple, take up from the start and the source volume the same space, but according to create snapshots after the upper business data, for real-time to take up the necessary storage space.

Rollback:

As shown in the preceding figure, the prerequisite for the rollback operation is to lock the source volume (suspend I/O operations on the logical address to be rolled back), and copy the data on the snapshot volume to the source volume by querying the mapping table.

Snapshot deletion:

For a snapshot that uses COW, the source volume stores complete real-time data. Therefore, when a snapshot is deleted, the snapshot volume and mapping table are directly destroyed, and data interaction with the source volume does not exist.

Redirect ROW while writing

Redirect-on-write (ROW) is also called write – time redirection.

After a snapshot is created, the snapshot system redirects write requests to the data volume to the storage space reserved for the snapshot and directly writes new data to the snapshot volume. When upper-layer services read a source volume, data before a snapshot is read from the source volume, and data generated after a snapshot is created is read from the snapshot volume.

Write operations:

Pictured above a brief example, after snapshot creation, if the upper business writing data to the wave source X, X in the cache line, snapshot system to judge X will be written to the logical address of the source volume, then set aside to write data X into the snapshot volume of corresponding logical address, at the same time, the source volume and snapshots of logical address write mapping table, namely writing redirection. We can see that the upper layer writes a block X to the source volume, and only one write occurs on the storage, except that it is redirected before the write.

Read:

After a snapshot is created, upper-layer services read data from the source volume in either of the following situations: 1) If the data is generated before the snapshot is created and is stored on the source volume, the upper-layer services read data from the source volume. 2) If the data to be read is generated after a snapshot is created, the upper-layer layer queries the mapping table and reads the data from the snapshot volume (read redirection).

After a snapshot is created, upper-layer services read data from the snapshot volume in either of the following situations: 1) If the data to be read is generated before the snapshot is created and stored on the source volume, upper-layer services query the mapping table and read data from the source volume. 2) If the data to be read is generated after a snapshot is created, the upper-layer layer directly reads the data from the snapshot volume.

As you can see, ROW snapshots also occupy the necessary storage space in real time based on the data generated by the upper-layer business after the snapshot is created.

Rollback:

After a snapshot is created, data generated by upper-layer services is written to the snapshot. Therefore, the rollback of a snapshot only cancels read redirection for the source volume. In plain English, no data operation is performed on the source volume. Upper-layer services only read data from the source volume (that is, data from the snapshot volume is not read from the source volume).

Snapshot deletion:

After a snapshot is created, data generated by upper-layer services is written into the snapshot. Therefore, before deleting a snapshot, copy the data in the snapshot volume to the source volume. If a snapshot is created at 18:00 for a source data, upper-layer services continue to generate a large amount of new data, snapshots are created at 19:00, and snapshots are created at 20:00… If multiple snapshots exist, data may be copied back to the source volume if the snapshots need to be deleted. As a result, the copyback may take a long time.

Compare the two technologies

As shown in the preceding table, copy-on-WRITE COW causes copy-on-write operations for each write. When a large number of writes are performed, the write performance of the source volume deteriorates, but the source volume is not affected. When a snapshot is deleted, the relationship between the snapshot and the source volume is removed, and the data on the snapshot volume is also deleted. ROW in each write only redirect the operation, the operating time is almost negligible, the source volume write performance is almost not affected, but read the source volume, then you need to determine the data is to create a snapshot before or after create snapshots, resulting in a large number of reading, the performance is affected by a certain, more deadly is, if the source volume has a multiple snapshots, when delete a snapshot, To ensure data integrity of the source volume, copy back all snapshot data to the source volume.

conclusion

The implementation principle of snapshot storage is briefly introduced. In fact, the snapshot feature is widely applied to many objects:

At present, mainstream manufacturers have made small changes to the above ROW and COW technologies in self-developed products, and some emerging snapshot technologies have been born, but there is no best snapshot technology in this industry. Technology serves business, and only local adaptation for business types can achieve the best utility.

Question and answer

Disappear stored procedure?

reading

Tencent Cloud CIS entry — Kubernetes deployment

Tencent Cloud API: Use Tencent Cloud API with Python (machine translation example)

Host migration practice sharing

Has been authorized by the author tencent cloud + community release, the original link: https://cloud.tencent.com/developer/article/1158686?fromSource=waitui

Welcome to Tencent Cloud + community or follow the wechat public account (QcloudCommunity), the first time to get more mass technology practice dry goods oh ~

Massive technical practice experience, all in the cloud plus community!