The introduction

Google has opened source a high-end image compression tool called Guetzli, and tried it out. It can compress an 89K image down to 33K.

$ ll guetzli/fe90f798bc8713c27bac5598a08b0c1aafded253_fw500.jpg resize-image/fe90f798bc8713c27bac5598a08b0c1aafded253_fw500.jpg
-rw-r--r-- 1 root root 33K May 31 16:58 guetzli/fe90f798bc8713c27bac5598a08b0c1aafded253_fw500.jpg
-rw-r--r-- 1 root root 89K May 31 16:58 resize-image/fe90f798bc8713c27bac5598a08b0c1aafded253_fw500.jpgCopy the code

I’m so happy to save so much money again. The only downside is that it takes too long to compress.

As a high-end programmer, I would not give up using it, so I took out my Periodic Task System and wrote a shared file System sharefs in order to be high-end enough. In this way, I created a distributed, reliable and stable graph making server. The architecture diagram is as follows:

Map the server architecture

• The user uploads an image and writes it to sharefs
• Upload the picture to Ali Cloud storage
• Do thumbnail also uploaded to Ali Cloud storage
• Compress the image generated above with Guetzli
• Upload the compressed picture to Ali Cloud storage to cover the previous file
• Sharefs caches the generated images for five days and then deletes them

Periodic Task System Overview

The periodic task system is used to do periodic tasks. Its architecture is shown as follows:

Timing task system architecture

• ClientSends tasks to the scheduled task system
• WorkerSubscribe to executable tasks (function name)
• When the time for a task is up, the task system assigns the correspondingWorkerperform
• WorkerReport execution results, such as success, failure, later execution

Do the thumbnail

import           Graphics.Image

height: : (Int.Int) - >Int
height (h, w) = width * h `div` w

image <- readImageRGB VU fileName
writeImage outFileName $ resize Bilinear Edge (height (dims image), width) imageCopy the code

I used Hip, haskell’s image processing library, which works well with multi-core cpus. I wrote Haskell-periodic just to use it.

The thumbnail Worker can be simply configured to form a new Worker, which can well meet the requirements of being compressed into different sizes.

Of course, you can also write in different languages. The scheduled task system supports clients or workers in multiple languages.

The compressed image

import           System.Process      (callProcess)
callProcess "guetzli" [ fileName, outFileName ]Copy the code

Pretty primitive, callProcess takes care of that

Uploading cloud Storage

To save trouble, just use github.com/aliyun/aliy… . Other cloud storage can be solved by writing an upload Worker.

Delete the cache

I use my own sharefs to implement file sharing between multiple servers. Of course, there must be problems with multiple files, so I need to delete expired files. Here directly use the function of the scheduled task system, time to directly delete the file is.

Fault-tolerant processing

Every Worker is likely to make errors in processing images. In case of errors in processing, the timed task system is required to try again after a period of time. When errors exceed a certain number of times, this task is abandoned.

Retry interval: Counter * (10 + counter) seconds.

conclusion

This is how high-end graphing servers are done. This system has been running stably in the production environment for a week.

Need help to leave a message or add wechat: Fisher_like

Follow me if you think the article is good and helpful to you.