Thanks to this article, I’m way ahead of my colleagues in development productivity

Welcome toTencent Cloud + community, get more Tencent mass technology practice dry goods oh ~

This article was published by Mr. Log Bridge in cloud + community column

introduce

If you have experience installing software from source code on a Linux server, you may encounter the make utility. This tool is mainly used to automatically compile and build programs. It allows the application author to easily lay out the steps needed to build that particular project.

Although make was created to automate software compilation, the tool is designed to be flexible enough to automate almost any task that can be done from the command line. In this tutorial, we’ll discuss how to readjust make to automate repetitive tasks that happen in sequence.

We’ll demonstrate this on Ubuntu, but it should work in a similar fashion on almost any Linux server.

Install the Make

Before we can start using make, we need to install it.

Although we can install it by name, it is usually installed with other tools that help you compile your software. We’re going to install all of these because they’re generally very useful. There is a package called “build-essential” that contains make and other programs:

sudo apt-get update
sudo apt-get install build-essential
Copy the code

You now have tools that allow you to leverage make in normal capacity.

Understand the Makefile

The main way to receive instructions with the make command is to use makefiles.

From the man page, we can see that make will find the file named GNUmakefile, then find the makefile, then find the makefile. It recommends that you use makefiles because GNU Makefiles are GNU specific commands, and makefiles don’t stand out.

Makefiles are directory-specific, which means make will search in the directory where it was called to find these files. Therefore, we should place the Makefile in the root of the task we are going to perform, or call the script we are going to write where it makes most sense.

In makefiles, we follow a specific format. Using the concepts of target, source, and command in this way:

target: source
	command
Copy the code

This alignment and formatting is important because make is picky. Here we discuss the format and meaning of each component:

target

Target is a user-specified name used to reference a set of commands. You can think of it like a simple function in a programming language.

Target, aligned on the left column, is a contiguous word (no Spaces) ending with a colon (:).

When calling make, we can specify target by typing:

make target_name
Copy the code

Make then checks the Makefile and executes the command associated with that target.

source

Source is a reference to a file or other target. They represent preparation or dependencies for the goals associated with them.

For example, you could make part of a make file look like this:

target1: target2
    target1_command

target2:
    target2_command
Copy the code

In this example, we can call target1 like this:

make target1
Copy the code

Make will then go to the Makefile and search for the “target1” target. It then checks to see if there is a specified source.

It finds the “target2” source dependency and temporarily jumps to that target.

From there, it will check if target2 lists any sources. It doesn’t, so it will continue to execute the “target2 command”. At this point, make reaches the end of the “target2” command list and passes control back to the “target1” target. It will then execute the “target1 command” and exit.

The source can be a file or the target itself. Use the file timestamp to see if the file has changed since the last call. Rerun the target if changes have been made to the source file. Otherwise, it marks the dependency as completed and continues to the next source, or command if this is the only source.

The general idea is that by adding sources, we can build a set of sequential dependencies that must be executed before the current target. You can specify multiple space-delimited sources after any target. You can begin to understand how to specify an elaborate sequence of tasks.

command

The reason for this flexibility with the make command is that the command part of the syntax is very open. You can specify any command to run under the target. You can add as many commands as you need.

The command is specified on the line following the target declaration. They are indented by a TAB character. Some versions of Make are flexible about how they indent parts of the command, but in general, you should stick to a single TAB to make sure make recognizes your intent.

Make treats each indentation under the target definition as a separate command. You can add as many indentation and commands as you need. Make will browse through them one at a time.

Before the command tells make to process them differently, we can place a few things:

• – : The dash before the command tells make not to abort if it encounters an error. For example, this action can be useful if you want to execute a command on a file (if it exists), and nothing if it doesn’t.
• @ : If you use the “@” symbol to boot a command, the command call itself is not printed to standard output. This is primarily used to clean up the resulting output.

Additional features

Some other features help you create more complex chains of rules in makefiles.

variable

Make identifies variables (or macros) that act as simple placeholders for replacements in makefiles. It is best to declare these at the top of the file. The name of each variable is fully capitalized. After the name, the equal sign assigns the name to the value on the right. For example, if we wanted to define the installation directory as /usr/bin, we could add at the top of the file:

INSTALLDIR=/usr/bin
Copy the code

Later in the file, we can refer to this location using the following syntax:

$(INSTALLDIR)
Copy the code

Span multiple lines

Another useful thing we can do is to allow commands to span multiple lines.

“\” to indicate crossing lines:

target: source
    command1 arg1 arg2 arg3 arg4 \
    arg5 arg6
Copy the code

This becomes even more important if you take advantage of some of the shell’s more programming features, such as if-then statements:

target: source
    if [ "condition_1"= ="condition_2"]; \then\
        commandto execute; \ anothercommand; \else\
        alternative command; \fi
Copy the code

This will execute the block as if it were a line of command. In fact, we could have written it as a single line, but it improves readability by splitting it up into something like this.

If you want to escape end-of-line characters, make sure there are no extra Spaces or tabs after the “\”, or you will receive an error.

File suffixes rule

Another feature you can use if you are doing file processing is file suffixes. These are general rules that provide a way to process files based on extensions.

For example, if you wanted to process all the.jpg files in your directory and convert them to.png files using the ImageMagick suite, we could use the following in a Makefile:

.SUFFIXES: .jpg .png

.jpg.png:
    @echo converting $< to $@
    convert $< $@
Copy the code

The first part is SUFFIXES. This tells make all the suffixes we will use in the file suffixes. The default includes suffixes commonly used to compile source code, such as “.c “and”.o “files, which do not need to be marked in this declaration.

The next section is the declaration of the actual suffix-rule. This basically takes the following form:

original_extension.target_extension:
Copy the code

This is not an actual target, but it will match any calls to files with the second extension and build them out of files with the first extension.

In our example, if we have “file.jpg” in our directory, we can call make to build a file named “file.png” :

make file.png
Copy the code

Make will see the PNG file in the.SUFFIXES declaration and look at the rules for creating the “. PNG “file. It then looks in the directory for the target file with “.png “replaced by”.jpg “. It will then execute the following commands.

The postfix rule uses some variables that we haven’t covered yet. These help replace different information depending on which part of the current process:

• $? : This variable contains a list of dependencies newer than the current target. These will be the goals that must be re-completed before the command under this goal can be executed.
• $@ : This variable is the name of the current target. This allows us to reference the file you are trying to make, even though this rule matches the pattern.
• $< : This is the name of the current dependency. For the suffix rule, this is the name of the file used to create the target. In our example, this will contain “file.jpg”
• $* : This file is the name of the current dependency that strips the matching extension. Think of this as an intermediate stage between the target and source files.

Create the conversion Makefile

We will create a Makefile that will perform some image processing and then upload the files to our file server so that our website can display them.

If you would like to follow along, download the ImageMagick conversion tool before you begin. These are simple command-line tools for manipulating images, and we’ll use them in our script:

sudo apt-get update
sudo apt-get install imagemagick
Copy the code

In the current directory, create a file called Makefile:

nano Makefile
Copy the code

In this document, we will begin implementing transformation goals.

Convert all JPG files to PNG

Our servers have been set up specifically to service.png images. Therefore, we need to convert any.jpg files to.png before uploading.

As mentioned above, postfix rules are a good way to do this. We’ll start with the.suffix statement, listing the format of the conversions between us:

.SUFFIXES: .jpg .png
Copy the code

After that, we can make a rule to change the.jpg file to.png. We can do this using the convert command in the ImageMagick suite. The convert command is simple and has the syntax:

convert from_file to_file
Copy the code

To implement this command, we need postfix rules to specify the format we start with and the format we end with:

.SUFFIXES: .jpg .png

.jpg.png:           ## This is the suffix rule declaration
Copy the code

Now that we have the matching rules, we need to implement the actual transformation steps.

Since we don’t know exactly what filename will be matched here, we need to use the variables we learned. Specifically, we need to reference $< as the original file and $@ as the file we want to convert. If we combine this with what we know about the convert command, we get the following rule:

.SUFFIXES: .jpg .png

.jpg.png:
    convert $< $@
Copy the code

Let’s add some functionality that tells us exactly what’s going on with the Echo statement. We will include the “@” symbol before the new command and the commands we already have to print the actual command at execution time:

.SUFFIXES: .jpg .png

.jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful!
Copy the code

At this point, we should save and close the file so that we can test it.

Get JPG files to the current directory. If you do not have the file at hand, you can enter the following content to obtain it from Tencent Cloud website:

wget https://ask.qcloudimg.com/raw/qctrain/yehe-b5f4bb2e421e9/1kp7y81up9.jpg
mv digitalocean-badge-blue.jpg badge.jpg
Copy the code

You can test that your make file works by asking it to create a badger.png file:

make badge.png
Copy the code

converting badge.jpg to badge.png using ImageMagick...
conversion to badge.png successful!
Copy the code

Make will go to the Makefile, see. SUFFIXES declaration in. PNG, then go to the matching SUFFIXES rule. Then run the listed commands.

Creating a file List

At this point, if we explicitly tell it we want the file, make can create a.png file.

It would be better if it just created a list of.jpg files in the current directory and converted them. We can do this by creating a variable that contains all the files to be converted.

The best way to do this is with the wildcard directive:

JPG_FILES=$(wildcard *.jpg)
Copy the code

We can specify a target with bash wildcards like this:

JPG_FILES=*.jpg
Copy the code

But there is a drawback. If there is no.jpg file, this will actually try to run the conversion command on a file named “*.jpg”, which will fail.

The wildcard syntax we mentioned above compiles a list of.jpg files in the current directory and does not set the variable to anything if it does not exist.

While we do this, we should try to handle slight changes to common.jpg files. These image files usually use the.jpeg extension instead of.jpg. To deal with these problems in a sensible way, we can change the name in the program to a.jpg file to make our processing lines simpler:

Instead, we’ll use the following two:

JPEG=$(wildcard *.jpg *.jpeg)     ## Has .jpeg and .jpg files
JPG=$(JPEG:.jpeg=.jpg)            ## Only has .jpg files
Copy the code

The first line compiles a list of.jpg and.jpeg files in the current directory and stores them in a variable called JPEG.

The second line references this variable and performs a simple name conversion, converting names ending in.jpeg in a JPEG variable to names ending in.jpg.

This is done with the following syntax:

$(VARNAME:.convert_from=.convert_to)
Copy the code

At the end of these two lines, we will have a new variable named JPG that will contain only the.jpg file name. Some of these files may not actually exist on the system because they are actually.JPEG files (no actual renaming has occurred). This doesn’t matter, because we’ll only use this list to create a new list of the.png files we want to create:

JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)
Copy the code

Now, we have a list of files that we want to request in the variable PNG. This list contains only.png file names because we did another name conversion. Now, each.jpg or.jpeg file in this directory is used to compile the list of.png files we want to create.

We still need to update. SUFFIXES declaration and SUFFIXES rules to reflect the.jpeg files we are now working with:

JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)
.SUFFIXES: .jpg .jpeg .png

.jpeg.png .jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful!
Copy the code

As you can see, we have added.jpeg to the suffix list and added another suffix match to our rule.

Create some Targets

We now have a lot in makefiles, but we don’t have any normal targets yet. Let’s fix this so that we can pass the PNG list to the suffix rule:

JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)
.SUFFIXES: .jpg .jpeg .png

convert: $(PNG)

.jpeg.png .jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful!
Copy the code

All of these new targets list our collection of.png file names as requirements. Then see if there is a way to take a.png file and use the suffix rule to do so.

Now we can use this command to convert all of our.jpg and.jpeg files to.png files:

make convert
Copy the code

Let’s add another target. Another task that is usually done when uploading images to a server is resizing them. Having the image properly sized will eliminate the need for the user to dynamically resize the image upon request.

ImageMagick’s Mogrify command can resize the image the way we want it to. Assume that the area our image will display on our website is 500px wide. We can convert this area using the following command:

mogrify -resize 500\> file.png
Copy the code

This will adjust any image larger than 500px wide to fit this area, but will not touch smaller images. That’s what we want. As a target, we can add the following rules:

resize: $(PNG)
    @echo resizing file...
    @mogrify -resize 648\> $(PNG)
    @echo resizing is complete!
Copy the code

We can add it to our file like this:

JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)
.SUFFIXES: .jpg .jpeg .png

convert: $(PNG)

resize: $(PNG)
    @echo resizing file...
    @mogrify -resize 648\> $(PNG)
    @echo resizing is complete!

.jpeg.png .jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful!
Copy the code

Now we can string these two goals together as a dependency on another goal:

JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)
.SUFFIXES: .jpg .jpeg .png

webify: convert resize

convert: $(PNG)

resize: $(PNG)
    @echo resizing file...
    @mogrify -resize 648\> $(PNG)
    @echo resizing is complete!

.jpeg.png .jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful!
Copy the code

You may notice that implicit resizing runs the same command as convert. We will specify both, though not always. Transformations can include more fine-grained processing in the future.

The Webify target can now convert images and resize them.

Upload the file to the remote server

Now that we have our images ready for the Web, we can create a target and upload them to a directory of still images on our server.

We can do this by passing the converted file list to SCP:

Our goal looks something like this:

upload: webify
    scp $(PNG) root@ip_address:/path/to/static/images
Copy the code

This will upload all our files to the remote server. Our file now looks like this:

JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)
.SUFFIXES: .jpg .jpeg .png

upload: webify
    scp $(PNG) root@ip_address:/path/to/static/images

webify: convert resize

convert: $(PNG)

resize: $(PNG)
    @echo resizing file...
    @mogrify -resize 648\> $(PNG)
    @echo resizing is complete!

.jpeg.png .jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful!
Copy the code

Clean up the

Let’s add a cleanup option to delete all local.png files after they have been uploaded to a remote server:

clean:
    rm *.png
Copy the code

Now we can add another target at the top that will be called after we upload the file to the remote server. That would be the most complete goal, and that’s what we want to default to.

To specify this, we will use it as the first available target. This will be used as the default. We will conventionally call it “all” :

JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)
.SUFFIXES: .jpg .jpeg .png

all: upload clean

upload: webify
    scp $(PNG) root@ip_address:/path/to/static/images

webify: convert resize

convert: $(PNG)

resize: $(PNG)
    @echo resizing file...
    @mogrify -resize 648\> $(PNG)
    @echo resizing is complete!

clean:
    rm *.png

.jpeg.png .jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful
Copy the code

With these final actions, if you enter a directory using makefiles and.jpg or.jpeg files, you can call make without any arguments to process the files, send them to your server, and then delete the.png files you uploaded.

make
Copy the code

As you can see, it’s easy to chain tasks together, and you can select a process to a point. For example, if you just want to convert files and need to host them on a different server, you can use the Webify target.

conclusion

---

At this point, you should have a good understanding of how to use makefiles. More specifically, you should know how to use make as a tool to automate most processes.

While it may be easier to write a simple script in some cases, makefiles are an easy way to establish a structured hierarchical relationship between processes. Learning how to use this tool can help simplify repetitive tasks. For more Makefile tutorials, please visit Tencent Cloud + community to learn more.

---

How To Use Makefiles To Automate Tasks on an Ubuntu VPS

Question and answer

The difference between BeautifulSoup and Scrapy crawler? reading

Tencent cloud database file back solution

Research and application of big data in education industry

Take a look at a clear view of context mapping

Deep Learning technology (NLP) with a PhD from Nanyang Technological University in Singapore

This article has been authorized by the author to Tencent Cloud + community, more original text pleaseClick on the

Search concern public number “cloud plus community”, the first time to obtain technical dry goods, after concern reply 1024 send you a technical course gift package!

Massive technical practice experience, all in the cloud plus community!