Some time ago, my website was suspected of being attacked. Today, I would like to take you to the scene of the accident and share my thoughts on accident analysis and prevention and control measures after the accident.

Since we have committed iniquity

Let’s start with how I found out about the site attack.

In general, in order to ensure the stable operation of online websites and background services, we need to add monitoring and alarm function to the project. When accidents occur, the system will send notification to the administrator in the first time.

As my project uses Tencent Yunyun development to deploy, it provides quota monitoring and alarm by default to prevent excessive resource consumption, which is very convenient.

But alarms alone are not enough. If something goes wrong, how can we analyze it? You have to give some clues to the troubleshooting.

Tencent cloud development provides monitoring and logging for cloud functions and cloud hosting by default. Without writing a line of code, you can see the operation information and detailed logs of resources, such as request time, IP address, request header information, etc., which is very convenient.

In addition, I added some logging and data reporting to the service during development, such as which user performed what action at which time. The more detailed the documentation, the easier it is to troubleshoot problems. Of course, meaningless content need not record, otherwise see the log when dense, hurt the eye and inefficient!

I’ve always thought of projects as my own children (although I don’t have any), so I check the monitors and logs every day to see how my “kids” are doing.

The monitoring metric I most often look at is the number of calls to the service, which largely reflects access to user traffic.

Normally, the graph of calls over time would look something like this, with no one looking at it at night and fairly smooth traffic during the day, with occasional small spikes:

But one day, I came across this curve, and what does this curve look like?

Yes, the Mediterranean has a long hair! In the vicinity of 25 minutes, the number of calls suddenly surge, we generally call this phenomenon as “traffic spike”, the monitoring chart of this unique as “burr”.

In most cases, having burrs is not a good thing. My first thought when I look at this curve is not, “Oh my God, the project is on fire?” “But” Oh my God, it’s under attack!”

Was he attacked or not? Who attacked me? I’m not really mad, am I?

With these questions in mind, let’s analyze it.

Analysis of the

You can’t just look at the graph, you have to look at the accident scene for clues.

Fortunately, the cloud developer helped us record the access log, select the time period when the incident occurred (based on 25 minutes, 5 minutes before and 5 minutes after), and then filter out the corresponding log.

For more flexible analysis, we exported the log to the local and opened it with spreadsheet software such as Excel.

Then, let’s analyze the log, first look at the log production time column, namely the time of the crime:

Did you notice that? Log production time is very even! About 3-4 per second.

From this point of view, there is a high probability that the service will not be accessed manually, but that the machine will automatically send requests at a certain frequency.

Take a look at the contents of the log. The structure of each log is as follows:

// Request time 2021-04-29T04.22:05.937752445z // IP stdout F 169.254.128.20 // Request header HEAD /webroot.bak HTTP/1.1\ // Response status code 200 0 // Request address http://www.code-nav.cn/webroot.bak // Request browser identity Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident / 5.0)Copy the code

The request time, request IP address and request address are the key information. Time has just been analyzed, let’s look at the request IP and address.

I directly searched the table globally for the above IP and found that all IP addresses are the same!

Well, that’s a relief. I think it was just one guy messing around.

Then I looked at the request address for several consecutive logs, something like this:

http://www.code-nav.cn/111.gz
http://www.code-nav.cn/111.tar.bz2
http://www.code-nav.cn/111.dat
http://www.code-nav.cn/111.bz2
http://www.code-nav.cn/222.tgz
http://www.code-nav.cn/222.gz
http://www.code-nav.cn/333.zip
...
Copy the code

Seeing “111”, “222” and “333”, I figured out that the attacker was scanning my website with dictionary enumeration in an attempt to find the background address.

The principle of attack is very simple, like when root was a child we tried to crack other people’s passwords, one by one crazy random try. But attackers often use some web-scanning tool to give a machine a dictionary of possible passwords to try out instead of a human. When the number and frequency of trials are high, it is called “blasting”.

Back to my fear of being dominated by my college cybersecurity class…

Based on the above analysis, the “attacker” should just use my site to practice, after all, the scanning frequency is not high, the duration is not long, of course, I hope so.

The prevention and control

It was harmless, but insulting! Let me fully aware of their own website in security is short weight. At least should be abnormal flow appear whether to give me alarm, send a short message what of it!

If you build your own server to deploy the website project, you need to access or develop a business monitoring and alarm system. Although there are many third-party systems on the Internet, such as Zabbix, Prometheus (AlertManager), Grafana, etc., you need to deploy and maintain them by yourself. Need certain manpower and material cost.

However, with Tencent Yunyun development, in addition to the above mentioned basic resource quota alarm, you can also flexibly customize various advanced alarm policies.

For example, to add a maximum number of calls to the “like” function, select “Cloud Function” as the alarm object:

Configure triggering conditions. For example, if the number of calls exceeds 100 within 5 minutes, the alarm is generated:

After that, you can configure the alarm recipient, alarm mode, and alarm time range. You can choose from various options, such as email, SMS, and wechat.

This way, you can add an alarm to every function with the smallest granularity, so that you know when something goes wrong the first time.

Product introduction

Cloud Development (TCB) is a cloud native integrated development environment and tool platform provided by Tencent Cloud. It provides developers with highly available, automatic and flexible expansion of back-end cloud services, including computing, storage, hosting and other serverless capabilities, which can be used for cloud integrated development of various end applications (small programs, Public account, Web application, Flutter client, etc.) to help developers build and manage back-end services and cloud resources in a unified manner, avoiding tedious server construction, operation and maintenance during application development. Developers can focus on the implementation of business logic, with lower development threshold and higher efficiency.

Open cloud development: console.cloud.tencent.com/tcb?tdl_anc…

The product documentation: cloud.tencent.com/product/tcb…

Technical documentation: Cloudbase.net? From =10004

Technical exchange group, latest information follow wechat public account [Tencent Yunyun Development]