Recently today while testing volcano Engine’s ImageX processing it was found to provide a capability called blind watermarking;

Blind Watermarking introduction:

Use method test:

Blind watermarking will be automatically added when a URL is placed to access. After adding, the information inside the picture will be extracted again;

Related articles:

— — — — — — — —

One, foreword

1.1. What is a dark watermark

Broadly speaking, dark watermarking can be understood as adding hidden marks to some carrier data, which are beyond the range that humans and machines can easily perceive. Compared with common bright watermarks, such as company logo in pictures and videos, watermarking texture in paper money, etc. Dark watermarks are transparent and invisible to most perceptual systems. Here are two examples.

1.1.1 Symbols hidden in white paper

For example, the figure below is the steganographic refractory paper of a chemistry institute of The Chinese Academy of Sciences in Shanghai. It can be seen that a pattern and letters are hidden in a seemingly ordinary white paper. This pattern and letter is a dark watermark. It can be used to secretly transmit information, do anti-counterfeit identification, etc.

1.1.2 Qr code hidden in the picture

The following example is probably less common. It is a steganographic question in the 2020 ByteCTF(Bytecdance Network Security Challenge) Misc. By hiding the watermark method, an interesting TWO-DIMENSIONAL code is embedded in the following color picture, and this two-dimensional code is invisible to the naked eye.

Dark watermarked images:

Hidden information in the image:

1.2 Common dark watermarking technology

This classification is based on transport carrier classification. Generally speaking, dark watermarks can be hidden in most multimedia transmission and storage carriers, such as pictures, videos, audio, emails, documents, etc., which are good carriers.

1.2.1 Image Watermarking

Image – based dark watermarking technology is one of the most mature dark watermarking, and there are many embedding methods. According to different embedding dimension, it can be subdivided into spatial watermark and transform domain watermark. Spatial watermarking can be simply understood as editing and embedding information directly to the pixel value of the decoded image. Transform domain watermarking is to convert the pixel information of an image to the transform domain, and then to the spatial domain after adding information in the transform domain. In this process, the spatial domain information will also be modified. Therefore, transform domain watermarking can also be understood as indirect spatial watermarking.

1.2.1.1 Spatial Watermarking

Directly select the pixel value of a specific location in the airspace and modify it to embed information. The difficulty of spatial domain watermarking lies in how to select watermark region in spatial domain and how to embed data in watermark block. According to the different selection methods of watermark region, it can be divided into the following types.

  • Least Significant Bits (LSB) watermark

This method is simple and crude, directly modify the pixel value of the image. Assume that the watermark carrier is an RGB image with color depth of 8bit, and the watermark information is a binary image.

  1. Watermarking process

Set the last 1bit (least significant) of the original image to zero, and assign the copyright information represented by binary to the last 1bit of the original image to realize the writing of copyright information. (The pixel value of the original image changes by 1 after writing)

  1. Watermarking process

Set the first 7bits (the highest 7bits) of the image to zero and lift the last 1bit (the lowest bit) to obtain copyright information.

  1. Algorithm profile

The computational complexity of this algorithm is relatively low. It has little effect on image visual effect; It has low robustness and is difficult to resist common watermarking attacks.

1.2.1.2 Transforming domain Watermarks

The watermarking in the transform domain will eventually modify the spatial data, which is different from the above. Instead of directly modifying the pixel value, the spatial data of the image is converted to the transform domain, and then the watermark information is written in a certain way. Finally, the data in the transform domain is converted back to the spatial value and the image information is regenerated.

DCT, DFT and DWT are the basic algorithms commonly used in transform domain watermarking. These three algorithms have different characteristics and can be used alone or cross-used.

  • Watermarking algorithm based on DCT

DCT discrete cosine transform (DCT) is a special DFT discrete Fourier transform (DFT), which has a very similar technique in the field of dark watermarking. Therefore, only the watermarking based on DCT transform is expanded here. The following figure shows the spectrum distribution of “dandelion” gray scale image after DFT and DCT transformation. It can be seen that the peak value distribution is different.

  1. What is the frequency domain transformation

The logic of frequency domain transformation can be understood in the following figure. The one-dimensional waveform of the “time domain” (which can be roughly understood as airspace) in the lower left corner can be determined by f1, F2, F3, F4… And so on multiple regular waveform superimposed, and these waveform corresponding to a fixed frequency, then they are projected to the lower right corner of the “frequency domain” to form another coordinate map. This process can be simply understood as the Process of the Fourier transform.

So what does it do in image processing?

The following four images are respectively the original image >> grayscale image of frequency domain after DCT transformation >> Clearing data in the upper left corner of frequency domain >> again converted into spatial images.

You can see that the converted image loses a lot of information, but you can still see the details of some hairs.

  1. How to write hidden watermark based on DCT

The embedding process of hidden watermark is as follows, and the framework is relatively simple. In practical application, different blocks and different frequency domain regions will be selected according to different scenarios, which need to be selected after a large number of experiments and theoretical precipitation.

  1. How to extract hidden watermark

There are many similarities between the extraction process and the watermarking process, but the difference is that it was written in the frequency domain before, and now it needs to be adjusted to read the information in the frequency domain and judge.

  1. Algorithm profile

This framework has strong generality, and can resist common attack means such as clipping, scaling and compression to a certain extent under the condition of reasonable selection of block and frequency domain position.

  • Watermarking algorithm based on DWT

There are relatively few pure watermarking algorithms only applicable to DWT, and most cases are DWT + DCT, DWT + SVD and other types of combination watermarking. The reason why DWT is used is that it can improve the robustness of watermark to a certain extent and has little impact on the picture quality.

  • The principle of

DWT here mainly refers to wavelet decomposition.

It can be seen that after DWT processing, the contour information (lower right corner) and detail information (upper left corner) of the picture can be filtered out.

1.2.2 Text Watermarking

Text watermarking refers to embedding hidden watermark information into text information. This is mainly inspired by handwriting. Each person’s writing style is different, and the strokes are also different. So by introducing a reader’s unique font file into a highly classified document, the document is essentially embedding that person’s identity information. Fonts can be modified in many different ways, but the following is just an example.

Font for User 1:

Font for User 2:

In this example, the word “is” is different, so you can tell which of the two the screenshot is from.

Algorithm brief review:

  • Compared with the above image-based watermarking, text font has stronger robustness, but it has a disadvantage that text watermarking is greatly affected by the number of words. When the text is small, it may not be enough to identify a person’s full identity.

  • There are many kinds of text watermarks, such as embedding information by adjusting text spacing and document format.

1.2.3 Audio Watermarking

Hidden audio watermarking refers to embedding watermark information in audio information beyond the audible frequency, which can not only cause little interference to the audio, but also embed watermark information in the audio.

Based on this feature of ear, a watermarking algorithm suitable for audio can be designed by combining the algorithm used in the image watermarking.

  • Audio LSB watermark

It is similar to the video LSB watermark, except that the object is audio instead. In order to embed watermark data in audio signal, the insensitive sampling value is replaced by binary bits.

  • Spread spectrum steganography

Spread-spectrum steganography (SPSS) is a technique of superimposing the whole spectrum coefficient of carrier audio by selecting appropriate embedding intensity after the secret information is modulated by pseudo-random sequence. The traditional transform domain includes FFT, DCT and DWT. With the development of technology, spread spectrum stegography based on MDCT and MCLT transform domain has been proposed in recent years.

  • Watermarking algorithm based on echo

Echo Hiding algorithm is a classical algorithm. The core idea is to write the watermark data into the carrier data by introducing echo, which makes use of the lag masking feature of HAS, that is, the weak signal becomes difficult to be sensed after the strong signal disappears, and it can not be detected by human ear within 50-200ms after the strong signal disappears.

1.2.4 Watermarking technology based on deep learning

In traditional watermarking methods, some transformations are usually used to embed the watermark information into the hidden space, and hand-designed features are used to embed the watermark to ensure the robustness of the watermark. With the rapid development of deep learning technology, people begin to use neural networks to embed watermarks, and use adversation learning to enhance the robustness of watermarks, so as to obtain robust encoders and decoders.

  • Deep learning image watermarking

It is mainly composed of three parts: preprocessing network (P), encoder (H) and decoder (R). Firstly, the secret information S is input into P to obtain the secret information feature graph. Then, the feature graph is spliced with the carrier image C and input into H, and finally the image C ‘after the embedded information is generated. When decoding, R is used to recover the information S ’embedded in C’.

The frame in the right image is similar to that in the left, except that S does not need to be preprocessed. If the secret message is a binary string, it’s 0 0 either repeated multiple times to match the size of C, or 0 0 0 Either from the shape of C to the size of C, spliced with C, typed into H for embedding.

  • Deep learning font watermarking

In FontCode, the author makes some small perturbations to the geometric features of English characters according to the watermark information to generate watermark characters. At the embedded end, the method of character disturbance is based on character structure, while at the extraction end, the watermark information is recovered by recognizing the disturbance pattern.

Character disturbance model includes character matching, manifold generation and font generation. Taking advantage of the diversity of existing fonts, the model matches each character in all characters individually, and then generates low-dimensional manifolds based on these high-dimensional contour features. Font manifold is a kind of mapping that uses the latent variable generation model of Gaussian process to form low-dimensional space, and it can also map back to higher-dimensional features by modifying the low-dimensional space, so as to infer and obtain new fonts.

Use CNN to identify which encoding text belongs to:

2. Introduction to application scenarios

The dark watermark is hidden in the material and cannot be perceived easily, which does not affect the normal use of the material, so it has many application scenarios.

2.1 Evidence of copyright video handling infringement

This is the most widely used scene of dark watermarking, and the process is relatively simple. By implicitly writing the unique identity of the author or manufacturer in copyrighted audio, video and pictures. In subsequent copyright disputes, if the watermark is extracted successfully, it will add a weight. The overall process is relatively simple and will not be repeated here.

The algorithm capabilities depend on in this application scenario vary according to the specific scenario:

  1. If it is only a simple crawling and handling, then the watermark can resist transcoding attack.
  2. If the material is deeply edited twice after handling, it requires that the watermark can resist zooming, clipping, compression and other attacks.
  3. There is also a difficulty, the final rights, may need to make the dark watermark legal recognition.

2.2 Trace the source of the leak

Nowadays, it often happens that the internal data screenshots of a factory are leaked. In this case, how to quickly locate the identity of the leak is difficult. Dark watermarking can solve this problem to a certain extent.

Manufacturers can add dark watermarks to sensitive pages or documents. The watermarks take place when the user is browsing the page, and the content is the user’s personal information. If a user takes screenshots of these pages or downloads them and spreads them maliciously, a dark watermark with his or her personal identity will follow the data. This makes it possible to quickly locate the leaking case through dark watermark. Thus it can trace the source of leakage and indirectly protect the data leakage.

Due to the complexity of leaked scenes and the diversity of data transmission chains such as screenshots and documents, it is difficult for single-dimensional dark watermarks to withstand the whole transmission process, and it is likely that the watermark information will be lost in the transmission process, resulting in the final failure of obtaining evidence. Therefore, it is necessary to design a multi-layer dark watermarking strategy to improve the robustness of watermarking without affecting the visual effect.

2.3 Material content infringement review

This can be understood as a branch of copyright protection. But this is where the platform protects users’ original videos from being stolen by other users within the platform. A more common scenario is video moderation platforms. The dark watermark can be used to protect the author’s original video from stealing and re-uploading in the platform.

This scene has high requirements on the comprehensive capability of dark watermarking:

  1. High robustness. It can resist video compression, zooming, clipping within frames, clipping between frames and other attacks.
  2. Low time delay watermarking algorithm. Because the watermarking process needs to be added in real time when the user uploads the video.
  3. Faster watermarking algorithm. Because the video review ultimately requires human participation to determine whether the infringement is real, and due to the limited time for the review of students, the extraction algorithm must complete the verification and extraction process of the entire video within ten seconds.
  4. Visual effects and various video metrics are demanding. After adding the watermark, the video must not make the video user value negative, which involves the requirements of the visual effect, bit rate and other indicators of the dark watermark.

2.4 tamper-proof

With the progress of financial informatization and the development of network and electronic commerce, electronic payment means has been paid more and more attention by the financial industry and has become one of the important means of payment. However, the screenshots of electronic payment bills have a high risk of being tampered with in the transmission process. In this process, the fragile watermarking system can be introduced and added when users generate electronic payment bills. When the receiver receives the bill, he checks the integrity of the dark watermark in the bill, so he can confirm whether the bill has been modified.

This watermark has the following characteristics:

  1. It is relatively fragile and will prompt watermark identification error after modification. But not so vulnerable to the spread of common social software.
  2. Wrong coordinate identification. Identify the area in which the data (watermark) has been modified.

2.5 Link Tracing

In some video dissemination business scenarios, it may be necessary to track the video dissemination process, but some external platforms cannot provide tracking services. Therefore, some other means are needed to track the whereabouts of data, and dark watermarking is a very ideal means. The dark watermark is added to the material before the video is put into use. When the material is played or used, the current node is marked according to the watermark information, so the data link can be traced according to the dark watermark.

The use of dark watermarking in link tracing requires the following characteristics: Luban requirements are general, mainly focused on CRF compression resistance.

Third, concluding remarks

Dark watermarking has a long history. Due to the popularity of short videos and the increasing awareness of copyright in recent years, dark watermarking is enjoying a new spring again. Under the new social background, dark watermarking is expected to play an increasingly important role in copyright protection and data protection.