preface

Many people like me often use grep command in daily life, but have never thought about what the function of grep command is. You might think of filtering, which is close. In fact, grep is a powerful text search tool.

For myself, there are two common grep scenarios: one is the filtering process, which will not be described here; The second is filtering logs, which is the focus of this paper.

The test file

I got a test log here

# app.log

2020-12-26 00:20:22 | INFO | GET /logout | rzflcymjduhtpivgbwea
2020-12-26 00:25:22 | DEBUG | GET /logout | hfkciozaywlbunpdrgsq
2020-12-26 03:21:22 | ERROR | GET /user/info | pwjulnfezbvxisokrtqm
2020-12-26 10:20:22 | ERROR | GET /logout | ilnquztfhcmgyxboksjp
2020-12-27 04:20:22 | INFO | GET /logout | uqdpcvbfnjstxgmzohea
2020-12-27 08:20:22 | DEBUG | GET /login | zknuobcmqlavwsdteypr
2020-12-28 00:20:22 | ERROR | GET /user/info | kqncawjeyobzvlhxtfdp
2020-12-29 00:20:22 | INFO | POST /login | jhmipunxfresbyvlckqg
Copy the code

1. Search in a single file

Format: grep < keyword > File name

1.1 Searching for error Logs

$ grep ERROR app.log
2020-12-26 03:21:22 | ERROR | GET /user/info | pwjulnfezbvxisokrtqm
2020-12-26 10:20:22 | ERROR | GET /logout | ilnquztfhcmgyxboksjp
2020-12-28 00:20:22 | ERROR | GET /user/info | kqncawjeyobzvlhxtfdp
Copy the code

Tip: The average error log is much larger, so you can write the error log to a new file for easy viewing

grep ERROR app.log > error.log

1.2 Searching for an Interface

$ grep /login app.log
2020-12-27 08:20:22 | DEBUG | GET /login | zknuobcmqlavwsdteypr
2020-12-29 00:20:22 | INFO | POST /login | jhmipunxfresbyvlckqg
Copy the code

Add: In addition to looking for error logs, I also encountered a situation where the company had a large number of nginx profiles and each port represented an application, so I could grep the port to find the specific application.

2. Search in multiple files

Format: grep < keyword > File 1 File 2

$ cp app.log app.log.1

$ grep ERROR app.log app.log.1app.log:2020-12-26 03:21:22 | ERROR | GET /user/info | pwjulnfezbvxisokrtqm ... (omitted)Copy the code

3. Advanced search

3.1 Searching for a Time range

#Accurate to the minute (0:20 to 0:29)
$ grep 'the 2020-12-26 00:2 [0-9]' app.log
2020-12-26 00:20:22 | INFO | GET /logout | rzflcymjduhtpivgbwea
2020-12-26 00:25:22 | DEBUG | GET /logout | hfkciozaywlbunpdrgsq


#Accurate to the hour (3 to 9)
$ grep 'the 2020-12-26 [3-9]' app.log
2020-12-26 03:21:22 | ERROR | GET /user/info | pwjulnfezbvxisokrtqm


#Accurate to the day
$ grep 'the 2020-12-2 [7-8]' app.log
2020-12-27 04:20:22 | INFO | GET /logout | uqdpcvbfnjstxgmzohea
2020-12-27 08:20:22 | DEBUG | GET /login | zknuobcmqlavwsdteypr
2020-12-28 00:20:22 | ERROR | GET /user/info | kqncawjeyobzvlhxtfdp
Copy the code

Note: You can also run the sed -n command to query logs in the format of sed -n ‘/ start time /,/ end time /p’. Note that the start time and end time must exist in the log file (if they do not exist, unexpected query results may occur).

$ sed -n '/2020-12-26 00:25:22/,/2020-12-27 04:20:22/p' app.log 
2020-12-26 00:25:22 | DEBUG | GET /logout | hfkciozaywlbunpdrgsq
2020-12-26 03:21:22 | ERROR | GET /user/info | pwjulnfezbvxisokrtqm
2020-12-26 10:20:22 | ERROR | GET /logout | ilnquztfhcmgyxboksjp
2020-12-27 04:20:22 | INFO | GET /logout | uqdpcvbfnjstxgmzohea
Copy the code

3.2 multistage lookup and operation (in multiple conditions at the same time) this is actually very common, use the pipeline operator | can, such as:

$ grep 'ERROR' app.log | grep 'user/info'        
2020-12-26 03:21:22 | ERROR | GET /user/info | pwjulnfezbvxisokrtqm
2020-12-28 00:20:22 | ERROR | GET /user/info | kqncawjeyobzvlhxtfdp
Copy the code

3.3 Multi-level lookup or operation (meeting one of the criteria) : Regular lookup is used here

$ grep -E 'ERROR | /login' app.log 
2020-12-26 03:21:22 | ERROR | GET /user/info | pwjulnfezbvxisokrtqm
2020-12-26 10:20:22 | ERROR | GET /logout | ilnquztfhcmgyxboksjp
2020-12-27 08:20:22 | DEBUG | GET /login | zknuobcmqlavwsdteypr
2020-12-28 00:20:22 | ERROR | GET /user/info | kqncawjeyobzvlhxtfdp
2020-12-29 00:20:22 | INFO | POST /login | jhmipunxfresbyvlckqg
Copy the code

3.4 Counting the number of rows: Counting the number of rows is very simple. Add the -c parameter

$ grep 'ERROR' app.log -c
3
Copy the code

3.5 Display line number: Display line number is also very simple, add -n parameter can be

$ grep 'ERROR' app.log -n
3:2020-12-26 03:21:22 | ERROR | GET /user/info | pwjulnfezbvxisokrtqm
4:2020-12-26 10:20:22 | ERROR | GET /logout | ilnquztfhcmgyxboksjp
7:2020-12-28 00:20:22 | ERROR | GET /user/info | kqncawjeyobzvlhxtfdp

Copy the code

Write in the last

Dear bosses, creation is not easy, but it needs to be honed and summarized constantly, welcome to pay attention to me, I am Yan Gan, I will share with you all kinds of practical programming knowledge and skills, your praise and attention is the biggest motivation for my progress and creation!!