Zeng Zhaohu

In May 2018, he joined Qunar and is now responsible for the development and construction of basic security attack and defense platform and daily security operation

background

High-risk vulnerabilities of Jar components such as Xstream, Jackson, and Fasjson emerge in an endlessly stream. Security groups push business lines to upgrade third-party Jar components N times a year, and each upgrade often involves hundreds of applications and services, bringing heavy burdens to both sides. In order to reduce the workload of the security group during Jar component upgrade and reduce the burden of the business line as much as possible, Qunar security group has carried out a lot of practices in the automated risk monitoring and upgrade of Jar components, and concluded a set of relatively complete solutions. Qunar Security Group is exploring and practicing automated risk monitoring and upgrading of Jar components.

The process is introduced

The essence of Jar component risk monitoring and upgrading is a workflow driven by risk intelligence, which mainly includes external security notification monitoring, Jar component asset collection, affected asset analysis, notification of line of business upgrades, and other processes. In the past, the escalation of Jar component vulnerabilities relied on security operators to manually wire each process, which was inefficient and even error-prone. As SOAR (Security Choreography automation and response) has attracted much attention in recent years, Qunar Security Group has also built on SOAR project. Relying on the capability of integrating events and connecting security services in SOAR, we have conducted reasonable choreography for the risk monitoring and upgrade scenarios of Jar components, achieving the effect of automation. Greatly improves the efficiency of safe operation. In addition, infrastructure colleagues provide automatic TCDEV upgrade service, which provides great convenience for line of business upgrade operations. The event flow is shown in the figure below:

The technical implementation

This section describes the technical implementation of security tools and services for each part of the SOAR tandem operation.

1. Safety notification monitoring

Security operation personnel get the vulnerability notice in the first time, and it is very important to evaluate and evaluate the vulnerability, respond quickly and promote it orderly. As early as 19 years ago, the security team realized the “security Vulnerability intelligent awareness system” with the help of the graduate project. The main functions of the system are: \

  • CVE, CNVD, and vulnerability risk alarms of well-known manufacturers
  • Vulnerability information is reintegrated
  • Grab poCS for vulnerabilities that have POC
  • Vulnerability information fuzzy matching associated Jar component asset library (SecDB), IM warning security operators

The key point of this system is that it will associate the Jar component asset library through fuzzy matching. After associating the asset library with the asset, IM will send early warning information to the security operation personnel for further risk assessment and subsequent UPGRADE process of Jar components. The system flow chart is as follows:

Vulnerability awareness platform, take Xstream as an example to grab the effect drawing:

2. Collect Jar assets

Safe asset collection is one of the essential basic capabilities for safe operation. Qunar Security Group has always achieved the best level of asset collection in the industry. Currently, we use HIDS as the main platform to efficiently collect host assets regularly and in real time by scheduling asset collection plug-ins on the Agent. Agent scheduling schematic diagram:The main implementation ideas of the Jar component asset collection plug-in are as follows:

  • Find the cataline.base list
items=$(ps aux | grep catalina.base | grep -v grep)
Copy the code
  • Obtain path information such as catalina.home and catalina.base
catalina_home=$(echo "$item" | tr ' ' '\n' | grep catalina.home | cut -d= -f2 | sort | uniq)
catalina_base=$(echo "$item" | tr ' ' '\n' | grep catalina.base | cut -d= -f2 | sort | uniq)
Copy the code
  • Get and information based on server.xml
  • Locate the WEB-INF/lib path based on the appBase or docBase
  • Enumerate the Jar packages in the WEB-INF/lib directory and extract the POM. properties information of each Jar package, so that you can collect assets. For example:
jar_version=$(echo "$pom_properties" | grep -m 1 -E '^version=' | awk -F'=' '{print $NF}' | tr -d '\n\r')
jar_groupid=$(echo "$pom_properties" | grep -m 1 -E '^groupId=' | awk -F'=' '{print $NF}' | tr -d '\n\r')
jar_artifactid=$(echo "$pom_properties" | grep -m 1 -E '^artifactId=' | awk -F'=' '{print $NF}' | tr -d '\n\r')
Copy the code

Through the above means, we can obtain the Jar package information on the host that the Java project depends on. Once the vulnerability breaks out, the associated application and Owner can respond quickly according to the above information. Taking Xstream as an example, the asset information collected is as follows:

3. SOAR

SOAR is the name of Security Orchestration, Automation and Response, which mainly focuses on the Security operation field. The Qunar security group built the SOAR project based on the secondary development of the StackStorm workflow engine. The security components and scripts were implemented in Python and YAML. The process in the Jar component automation risk monitoring and upgrade scenario is shown below:From the workflow diagram, we can see:

(1) Manual intervention is required after the warning information is issued by the security notification monitoring service. The security operation personnel will determine whether to start the upgrade process. If yes, fill in the configuration vulnerability information and start the upgrade process. Otherwise, ignore the alarm

A) Asset list generation: Generate a list of affected assets through logical filtering based on the configured version information. At the same time, mark the extranet and extranet for executing different priority policies. B) Associate Appcode: Obtain the Appcode of the affected host using the Portal API. C) Associated Owner: Obtain the Owner of the affected host using the Portal API. D) Association technology TL: The Owner’s technology TL is associated with ISAPI employee information (the technology TL acts as a security interconnect and performs top-down vulnerability promotion)

(3) Then the asset list will be provided to TCDEV, TCDEV will take over the application that can be automatically upgraded, and the rest will continue to be notified by the security responsibility of the line of business technology TL upgrade

Xstream vulnerability upgrade example, configure vulnerability information, and start automatic associated information upgrade notification process:

Xstream security notification example, using the internal IM notification technology TL to perform upgrade tasks:

4. TCDEV Automatic upgrade service

In a typical company, a risk event is notified to the person responsible for the event and the entire event process is completed, and then periodic notification escalations are performed. However, within Qunar, the automatic upgrade service developed based on TCDEV can greatly relieve the risk component upgrade pressure in the line of business. TCDEV automatic upgrade service can help line of business to automatically upgrade Jar components. 50% of current applications can be automatically upgraded, 30% of applications can be upgraded through the one-click upgrade service provided by TCDEV (requiring line of business development to assess the risk), and the other 20% of applications can implement the traditional upgrade policy of security group. TCDEV automatic upgrade details are as follows:

  • If the application has been upgraded with TCDev 4.x and the application has been connected to thanos automated testing, TCDEV will take over the upgrade and contact the service for confirmation (50% applications).

  • For applications that have been upgraded to TCDev 4.x but are not covered by the automated test, click “TCBOM Upgrade” on the portal to quickly complete the upgrade (30% of the applications) \

  • If tcDev 4.x has not been upgraded, you are advised to manually upgrade TCDev to 4.0.x (20% applications).

conclusion

These are all aspects of the Jar component automation risk monitoring and upgrade practice. There is still room for improvement in the overall process, such as the need for human intervention in vulnerability assessment and automatic TCDEV upgrade services. In addition, TCDEV automatic upgrade service value is great, due to less information, did not touch the principle of implementation, hope that the students of infrastructure can write an article to introduce it. Due to the limited level, there are many flaws in the article, also ask you to correct.