Original: Coding diary (wechat official ID: Codelogs), welcome to share, reprint please reserve the source.

Introduction to the

There are many packet capture tools in Linux, such as ngrep, tcpdump, and tshark. There are many usage and application scenarios of these tools.

Linux command gap-Getting Started Linux command gap-text Processing Linux command gap-Software Resource Observation Linux command gap-Hardware resource observation Linux command gap-Profiling tools Linux command gap-dynamic tracking tools Linux command gap-Understanding system load What is %nice in the Linux command gap-top

ngrep 

Ngrep is a packet capture tool, which directly displays the captured packet data in the form of text. It is very suitable for packet capture analysis (such as HTTP and MySQL) when packet data contains text, as follows:

Catch a HTTP request

Sometimes we run into situations where a server provides an HTTP interface for the caller to use, and the caller passes in a parameter value, and the server gets a different result! In this case, either the caller has a problem with the way the request is made, or the server has a request interceptor that causes the problem. How can you tell which is the problem? Use ngrep to check the contents of the package, as follows:

-d any Captures data packets of all nics
# -w byline is used for HTTP packet capture
$ ngrep -d any -W byline port 8080
Copy the code



As shown above, ngrep displays non-visible characters as., so the HTTP request above has one after each line.Is the HTTP newline character\r\nIn the\r“, and the T in front indicates that this is a TCP packet.

Grab the SQL

Select (select) from network packets
The two packets following the matching packets are also displayed
$ ngrep -d any -A 2 'select' port 3961
Copy the code



You can find that the mysql back package has.def.We can use this to determine the mysql response package.

Ngrep can print the interval time of the two packets with the -t parameter, so that we can roughly see the time of SQL query according to the interval time of packet return, as follows:

# -w single allows package data to be displayed on a single line, making it easy for text tools like AWK to process
# -s 200 The maximum packet size is 200 bytes. Redundant data is not displayed
# -t Prints the interval between two packets in seconds
Part of the script logic is to keep capturing packets until you find that the packet return is slower than 1 second
$ ngrep -d any -W single -s 200 -T 'select|def' port 3961 \
    | awk '{print} $1~/T/ && /.def./ && $2>1 && $2<1000 {exit(0)}'
Copy the code

tcpdump

Tcpdump is a universal packet capture tool. It is used to capture network packet data of various protocols and then analyze the data using wireshark.

Capture port 3961 network packet

# -c 10000 Indicates that a maximum of 10,000 packets can be captured
$ tcpdump -i any -s 0 -c 10000 tcp and port 3961 -w ./target.cap

-g 600 indicates that 600s saves a captured packet file to avoid a single file being too large
$ tcpdump -i any -s 0 -G 600 tcp and port 3961 -w ./target_%Y_%m%d_%H%M_%S.pcap
Copy the code

Wireshark analysis

Grab the target. The cap is mysql network packet data, it is binary, not directly to see, you need to use wireshark analysis, download address is as follows: www.wireshark.org/download.ht…

By default, the Wireshark uses port 3306 to connect to the TCP protocol. The wireshark uses port 3961 to connect to the TCP protocol. The Wireshark uses port 3961 to connect to the TCP protocol. Wireshark: use MySQL to parse port 3961 packets as follows:

A. first,decode as...MySQL protocol is used for port parsing:



.



B. the inputmysql.query contains "id=24218", check the SQL containsid=24218As follows:

Slow network or slow back-end processing?



You are advised to add the following two columns to the Wireshark to facilitate time analysis:

  • TcpDelta = tcp.time_delta: indicates the time difference between the current packet and the previous one in the current TCP connection.
  • Ack_rtt = tcp.analysis.ack_rtt: indicates the time difference between ACK packets in TCP and their packets.

As follows, find the package that takes the most time:



.



.



.



Found forThe select sleep (2.0)If the network is slow, the ack will be slow as well.

tshark

Tshark is the cli version of the Wireshark tool. The methods used in tshark are similar to those in Wireshark.

# -i any Fetch data from any network adapter
# -f specifies which protocol to capture, which port
# -d decode aS to decode mysql
# -y Displays filters similar to those in Wireshark
# -t specifies the data output format. Fields indicates tab-separated format
# -e Specifies the output field
# -e header=y Specifies the output header line
$ sudo tshark -ni any -f 'tcp and port 3961' -d 'tcp.port==3961,mysql' \
    -T fields  -e frame.number -e frame.time_epoch -e frame.time_delta_displayed  \
    -e ip.src -e tcp.srcport -e tcp.dstport -e ip.dst -e tcp.stream -e tcp.len -e tcp.nxtseq \
    -e tcp.time_delta -e tcp.analysis.ack_rtt \
    -e _ws.col.Info -e mysql.query -E header=y > packets.tsv

View captured packet data$ csvlook -It packets.tsv | less -iSFX | frame.number | frame.time_epoch | frame.time_delta_displayed | ip.src | tcp.srcport | tcp.dstport | ip.dst | tcp.stream | tcp.len | tcp.nxtseq | tcp.time_delta | tcp.analysis.ack_rtt | _ws.col.Info | mysql.query | | ------------ | -------------------- | -------------------------- | --------- | ----------- | ----------- | --------- | ---------- | ------- | ---------- | -------------- | -------------------- | -------------------------------------------------------------------------------- | -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - | | | | 1639287428.374886956 1 0.000000000 | 127.0.0.1 39262 | | 3961 | 127.0.0.1 | | 0 5 6 0.000000000 | | | | Request Ping | | | | 2 1639287428.375043740 0.000156784 | | 127.0.0.1 3961 | | 39262 | 127.0.0.1 | | 0 0 1 | | | | 0.000156784 0.000156784 3961 - 39262 (ACK) Seq = 1 ACK = 6 Win = 512 Len = 0 696789627 TSecr TSval = = 696789627 | | | 3 | | | 0.000225138 1639287428.375268878 127.0.0.1 3961 | | 39262 | 127.0.0.1 | 0 11 12 0.000225138 | | | | | Response OK | | | | | 1639287428.375289961 4 0.000021083 | 127.0.0.1 39262 | | 3961 | 127.0.0.1 | | 0 0 | | | | 0.000021083 0.000021083 6, 39262-3961 / ACK Seq = 6 Ack = 12 Win = 512 Len = 0 696789627 TSecr TSval = = 696789627 | | | | | | 0.000596178 1639287428.375886139 5 127.0.0.1 39262 | | 3961 | 127.0.0.1 55 0.000596178 | | 61 | | 0 | | Request Query | select id from app_log al order by desc idlimit1 | | | | | 0.000020264 1639287428.375906403 6 127.0.0.1 3961 | | 39262 | 127.0.0.1 12 0.000020264 | | | | | 0 0 0.000020264 | 3961-39262 / ACK Seq = 12 ACK = 61 Win = 512 Len = 0 696789628 TSecr TSval = = 696789628 | | | | 7 1639287428.377206294 0.001299891 | | 127.0.0.1 3961 | | 39262 | 127.0.0.1 | 0 76 | | 88 | | 0.001299891 | Response | | | 8 1639287428.377226652 0.000020358 | | | 127.0.0.1 39262 | | 3961 | 127.0.0.1 | | 0 0 | 61 | | 0.000020358 0.000020358 | 39262-3961 / ACK Seq = 61 ACK = 88 Win = 512 Len = 0 696789629 TSecr TSval = = 696789629 | | | | | 1639287428.378921659 9 0.001695007 | 127.0.0.1 39262 | | 3961 | 127.0.0.1 | 0 0.001695007 | | 84 | | 145 | Request Query | select id,log_info,create_time,update_time,add_time from app_logwhereId = 27371 | | | | | 0.000020725 1639287428.378942384 10 127.0.0.1 3961 | | 39262 | 127.0.0.1 | | 0 0 | 88 | | 0.000020725 0.000020725 | 3961-39262 (ACK) Seq = 88 ACK = 145 Win 512 Len = = 0 TSval = 696789631 TSecr = 696789631 | | | | 1639287428.380450661 0.001508277 | | 127.0.0.1 3961 | | 39262 | 127.0.0.1 | 0 412 | | 500 | | 0.001508277 | Response | | | | | | 0.000020975 1639287428.380471636 12 127.0.0.1 39262 | | 3961 | 127.0.0.1 | | 0 0 | 145 | | 0.000020975 0.000020975 | 39262-3961 (ACK) Seq = 145 ACK = 500 Win 509 Len = = 0 TSval = 696789633 TSecr = 696789633 | | | | 13 1639287430.051942681 1.671471045 | | 127.0.0.1 39262 | | 3961 | 127.0.0.1 22 1.671471045 | | 167 | | 0 | | Request Query | select sleep (2.0) | | | | | 0.000030080 1639287430.051972761 14 127.0.0.1 3961 | | 39262 | 127.0.0.1 | | 500 | | 0 0 | | 0.000030080 0.000030080 in 3961-39262 (ACK) Seq = ACK = 167 Win 500 = 512 Len = 0 696791304 TSecr TSval = = 696791304 | | | | 15 1639287432.053394353 2.001421592 | | 127.0.0.1 3961 | | 39262 | 127.0.0.1 | 0 65 | | 565 | | 2.001421592 | Response | | | | | | 0.000126321 1639287432.053520674 16 127.0.0.1 39262 | | 3961 | 127.0.0.1 | | 0 0 | 167 | | 0.000126321 0.000126321 | 39262-3961 (ACK) Seq = 167 ACK = 565 Win 512 Len = = 0 696793306 TSecr TSval = = 696793305 |Copy the code

In addition, Tshark can directly analyze packet data captured by tcpdump as follows:

# -y is similar to the display filter in Wireshark
$ tshark -d 'tcp.port==3961,mysql' -Y 'mysql.query contains "id=21"' -r target.cap -T fields -e frame.number -e mysql.query
5       select id,log_info,create_time,update_time,add_time from app_log where id=21527
13      select id,log_info,create_time,update_time,add_time from app_log where id=21518
25      select id,log_info,create_time,update_time,add_time from app_log where id=21007
52      select id,log_info,create_time,update_time,add_time from app_log where id=21505

The # tshark package also comes with an editcap command
A time range that can be used to intercept captured packet data
$ editcap -F libpcap -A "The 2013-07-20 23:00:00" -B "The 2013-07-20 23:20:00" input.pcap output.pcap
Copy the code

Pcap – filter with wireshark – filter

Ngrep, tcpdump, and tshark-f all use the pcap-filter syntax, which is used to filter captured network packets. The following lists common pcap-filter syntax used in tcpdump.

Capture packets from the specified host$tcpdump -ni any host 210.27.48.1Fetching TCP packets from host 210.27.48.1 and port 23$tcpdump -ni any TCP SRC host 210.27.48.1 and port 23TCP packets destined for port 80 or 8080 from a host on network 10.234.10.0/24$tcpdump -ni any TCP SRC net 10.234.10.0/24 and DST port'(80 or 8080)'
Capture the TCP RST packet
$ tcpdump -ni any -s0 tcp and 'tcp[13] & 4 ! = 0 ' -vvv
Capture the TCP FIN packet
$ tcpdump -ni any -s0 tcp and 'tcp[13] & 1 ! = 0 ' -vvv
Copy the code

Detailed grammar can man pcap – filter to view, or access: wiki.wireshark.org/CaptureFilt…

The Wireshark-filter syntax is used to filter and analyze captured data packets. The following lists common syntax used by Tshark to filter and analyze captured data packets.

# filter packets from specified host and specified port
$ tshark -Y 'IP. Addr = = 127.0.0.1 and TCP. The port = = 3961'
Parse to HTTP and use HTTP field filtering
$ tshark -d 'tcp.port==8080,http' -Y 'http.request.uri contains "/get"'
SQL > select * from mysql
$ tshark -d 'tcp.port==3961,mysql' -Y 'mysql.query contains "id=21"'
Copy the code

Detailed grammar can man wireshark – filter to view, or access: wiki.wireshark.org/DisplayFilt…

conclusion

Using ngrep, tcpdump, and tshark to analyze network problems is very helpful. You must try them in your work and be familiar with their usages.

Telnet, nslookup, curl, and so on. If you want to set the connection idle time, do not set the connection idle time. I got it. Why can’t you?

Content of the past

Awk is really a magic tool for Linux text command tips (top) Linux text command tips (bottom) character encoding solution