This article is published by netease Cloud.
Alexander Polyakov is cTO and co-founder of ERPScan, president of EAS-SEC and evangelist for SAP network security.
Quite a few articles have been written about machine learning and its ability to protect us from cyber attacks. However, it is also clear to separate the ideal from the reality and see what machine learning (ML), deep learning (DL) and artificial intelligence (AI) algorithms can do in network security.
First of all, I have to disappoint you, because we have to admit that machine learning is not going to be the silver bullet for cybersecurity, even though it’s making great strides in image recognition or natural language processing. There will always be people trying to find problems in our systems and trying to get around them. To make matters worse, these advanced technologies are also being used by hackers, who can use machine learning for their purposes, for example.
Machine learning can not only help us complete typical ML tasks, including regression (prediction), classification, clustering, recommendation. ML can also solve problems with varying efficiency for a variety of requirements, depending on the algorithm you choose. Now, we’ll use machine learning to solve typical cybersecurity tasks.
According to Gartner’s PPDR model, all security tasks can be divided into five categories: prediction, Prevention Detection, response and monitoring. More precisely, they can be used at technical layers such as network (network traffic analysis and intrusion detection), endpoint (anti-malware), application (WAF or database firewall) or user (UBAs, anti-fraud).
Now, let’s look at an example of how current machine learning approaches can be applied to network security tasks.
A, regression
Regression is a simple task, in other words prediction is a simple task. We want to use what we know about existing data to make predictions about new data, the simplest example being house price forecasts. In network security, it can be used for tasks such as user behavior analysis and fraud detection. Network traffic analysis is another good option for using machine learning. As for the technical aspects of regression, various types of recursive neural networks work best.
Second, the classification of
The classification problem is also simple. If you have two stacks of pictures, say of a dog and a cat, you can easily put the new picture on top of the corresponding one, which is often called supervised learning. We know exactly what we are looking for and put them in the right place. So how to apply machine learning classification algorithm to network security? Suppose we want to detect malicious activity on different layers. For the network layer, we can apply it to the intrusion detection system (IDS) and identify different types of network attacks, such as scanning, spoofing and so on. At the application level, we can apply it to WAF and detect OWASP Top 10 attacks. At the end layer, we can classify software into categories like malware, spyware, and ransomware. Finally, at the user level, it can be applied to anti-phishing solutions that tell us whether a particular email is legitimate or not. Technically, algorithms such as SVM or random forest and better simple artificial neural networks or convolutional neural networks can solve these tasks.
Third, clustering
The idea of clustering and classification to solve network security problems is basically the same, with only one major difference: we do not know any information about the data class. Furthermore, we don’t know if the data can be classified. This is called unsupervised learning. We don’t get involved in the data annotation process and leave all the tasks to the machine, which sounds like a very interesting experiment.
I find that one of the best tasks for clustering is forensic analysis — malware analysis solutions (malware protection) can perform this to separate legitimate files from outliers when we don’t know what’s going on and categorize all activity to find outliers. Another interesting area where clustering can be applied is in user behavior analysis. In this case, application users gather together and can see if they belong to a particular group. Provide effective network security solutions according to the group they belong to.
Fourth, to recommend
Recommendation system is a very famous system in the Internet era. For example, when we all use Netflix and SoundCloud, they’ll recommend movies or songs they think you like based on your movie or music preferences. The same idea can be applied to network security, where it can be used primarily for event response. If a company is faced with a series of events and offers various types of responses, the system can learn which type of response to recommend for a particular event. Risk management solutions can also benefit because they can automatically assign risk values to new vulnerabilities or build misconfigurations based on their descriptions. Many algorithms have emerged for recommendation tasks, the latest of which are based on restricted Boltzmann machines and their newer versions, such as deep belief networks.
Five, the conclusion
In addition to the security areas I mentioned, there are many security areas where machine learning can be applied. Machine learning is by no means a perfect solution if you want to protect your systems, but at the same time, it will become standard in cyber security in the near future, as hackers begin to use machine learning as a means of attack.
What can machine learning, deep learning, and AI algorithms do for network security?
That netease cloud yi Shield is how to use artificial intelligence? Two more dry articles to share:
The old ways of dealing with fleece gangs don’t work, but some companies still do. How do they do it?
2. Tracking predators: How underground ash kills Startups
Understand netease Cloud:
The official website of netease Cloud is www.163yun.com/
New user package: www.163yun.com/gift
Netease Cloud community: sq.163yun.com/