Chapter 1 analysis and design of billion controllable system
JavaEE Elite Advanced Course study notes to remove code: XB2K
Learning goals
- Understand the application field and development status of the Internet of things
- Can name billions of controllable core functions
- Able to draw a controllable system architecture diagram
- Be able to complete the preparation of a controllable environment and understand the functional structure of a controllable environment
- Complete the equipment management related function development
1. Industry analysis of the Internet of Things
1.1 What is Internet of Things
Internet of Things (Internet of Things, abbreviated: IoT) originated in the media field and is the third revolution of information technology industry. The Internet of Things refers to connecting any object to the network through information sensing equipment according to the agreed protocol. The object can exchange and communicate information through the information transmission media, so as to realize intelligent identification, positioning, tracking, supervision and other functions. V(cmL46679910)
There are three key technologies in the application of the Internet of Things, namely, the perception layer, the network transmission layer and the application layer.
Definition of Chinese Internet of Things:
The most concise definition: Internet of Things (Internet of Things) is a network based on the Internet, traditional telecommunications network and other information carriers, so that all ordinary physical objects can be independently addressed to achieve interconnection. It has three important characteristics: common object device, autonomous terminal interconnection and universal service intellectualization.
Four concepts emerge in the figure above, which we explain here:
The integration of informatization and industrialization is the high-level and deep combination of informatization and industrialization. It means that informatization drives industrialization and industrialization promotes informatization and takes a new road of industrialization. The core of the integration of the two is information support and the pursuit of sustainable development mode.
M2M, Machine to Machine, is the transmission of data from one terminal to another terminal, that is, the dialogue between machines.
Radio Frequency Identification (RFID) is short for Radio Frequency Identification. Its principle is non-contact data communication between the reader and the tag to achieve the purpose of identifying the target. The application of RFID is very extensive, typical applications are animal chip, car chip alarm device, access control, parking control, production line automation, material management.
Sensor network is short for sensor network, sensor network (WSN) is the collection of computer, communication, network, intelligent computing, sensors, embedded systems, microelectronics and other fields of overlapping comprehensive emerging discipline, it will be a lot of variety sensor nodes (integrating sensing, acquisition, processing, send and receive) composed of autonomous network, realize the dynamic intelligent collaborative perception of the physical world.
As can be seen from the figure above, the Internet of Things covers the four areas mentioned above.
Understand the Internet of things in one sentence
All items are connected to the Internet through information sensing equipment for information exchange, namely, the object phase information, in order to achieve intelligent identification and management.
history
The concept of the Internet of Things was called the Sensor network when it was first proposed by China in 1999. The Chinese Academy of Sciences started the research and development of sensor networks as early as 1999. Compared with other countries, China’s technology research and development level is in the forefront of the world, with the same development advantage and significant influence. On November 27, 2005, at the Information Society Summit held in Tunis, the International Telecommunication Union (ITU) issued the ITU Internet Report 2005: The Internet of Things, formally proposing the concept of the Internet of Things. On August 24, 2009, Wang Jianzhou, president of China Mobile, also mentioned the concept of the Internet of Things in his public speech in Taiwan. Zhu Hongren, chief engineer of the Ministry of Industry and Information Technology, said at the China Industrial Operation 2009 summer report that the Internet of Things is a new concept and there is no established and universally accepted concept until 2009. In general, he said, the “Internet of Things” refers to a new technology that connects sensors with the existing “Internet.” The Internet of Things is an “Internet of Things” that covers everything in the world based on the computer Internet and utilizes RFID, wireless data communication and other technologies. In this network, objects (goods) can “communicate” with each other without human intervention. Its essence is the use of radio frequency automatic identification (RFID) technology, through the computer Internet to achieve automatic identification of goods (goods) and information interconnection and sharing. The advent of the Concept of the Internet of Things has broken traditional thinking. The old thinking has been to separate physical infrastructure from IT infrastructure, airports, roads, buildings on the one hand, data centers, PCS, broadband, etc., on the other. In the age of the Internet of Things, reinforced concrete and cables will be integrated with chips and broadband into a unified infrastructure. In this sense, infrastructure is more like a new earth. Therefore, some insiders believe that the Internet of Things and smart grid are organic components of a smart earth.
1.2 Internet of Things application fields
1, smart home Smart home is the use of advanced computer technology, the use of intelligent hardware (helium-krypton wifi, Zigbee, Bluetooth, NB-iot, etc.), Internet of things technology, communication technology, and furniture life of various subsystems organic combination, through overall management, so that home life more comfortable, convenient, effective, and safe. Smart home mainly includes smart speaker, smart lamp, smart socket, smart lock, smart thermostat, sweeping robot and so on.
Intelligent transportation is a comprehensive transportation system that effectively integrates intelligent sensing technology, information network technology, communication transmission technology and data processing technology represented by the Internet of Things, The Internet and cloud computing, and applies them to the entire transportation system to play a role in a larger space and time range [2]. Intelligent transportation is a new mode of large-scale transportation development with intelligent road network, intelligent travel, intelligent equipment, intelligent logistics and intelligent management as its important content and characterized by highly integrated information technology and comprehensive application of information resources. Depends on Mr Dimon technology in cloud computing, Internet of things, big data, financial science and technology in the areas of development experience and strong technical accumulation, three-year pour to build China’s first a set of network about car, parking, car rental, car finance wisdom, wisdom, and other travel domain business model innovation in the integration of high-end intelligent traffic solutions for [3].
4, smart grid, smart grid is built on the basis of traditional grid set sensing, communication, computing, decision and control for the integration of comprehensive number of composite system, through access to grid nodes in each layer of resources and equipment running status, control and manage the hierarchical and electricity distribution, realize the energy flow, information flow and high integration of the business flow, Improve the stability of power system operation, in order to maximize the efficiency of equipment utilization, improve safety and reliability, energy conservation and emission reduction, improve the quality of power supply, improve the utilization efficiency of renewable energy.
Smart City smart city is to use information and communication technology to sense, analyze and integrate key information of the core system of urban operation, so as to make intelligent response to various needs including people’s livelihood, environmental protection, public security, urban services, industrial and commercial activities. Its essence is to use advanced information technology to achieve smart urban management and operation, and then create a better life for people in the city, and promote the harmonious and sustainable growth of the city. With the continuous development of human society, cities will carry more and more people in the future. At present, China is in the period of accelerated urbanization, and the problem of “urban disease” in some areas is increasingly serious. In order to solve the problem of urban development and achieve sustainable urban development, the construction of smart city has become an irreversible historical trend of urban development in the world today. Smart city construction has been carried out in many areas at home and abroad, and has achieved a series of achievements, such as smart Shanghai, smart Dual Flow; Foreign countries such as Singapore’s smart Nation Program and South Korea’s U-City Program.
5, other fields: intelligent vehicles, intelligent buildings, intelligent water, intelligent business, intelligent industry, safe city, intelligent agriculture, intelligent security, intelligent medical treatment, etc.
1.3 Development status of the Internet of Things
The consumer IOT boom is still in its infancy
The Internet of Things connects things to things and people to people through related devices.
(1) Scale: The scale of the global Internet of Things industry grew from $50 billion in 2008 to only $151 billion in 2018, with a compound annual growth rate of 11.7%. The scale of China’s Internet of Things industry reached 1.150 billion yuan in 2017, further accelerating since 2011, with a compound annual growth rate of 26.9% from 2009 to 2017, making China’s Internet of Things development faster than the global average.
(2) Penetration: The penetration rate of the global Internet of Things industry has more than doubled to 12% in 2013 and 29% in 2017, and it is expected that more than 65% of enterprises and organizations will apply Internet of Things products and solutions in 2020. In recent years, China’s Internet of Things market has been expanding, from 365 billion yuan in 2012 to 1160.5 billion yuan in 2017, with a compound annual growth rate of up to 25%.
2012-2017 Market size of Internet of Things in China (100 million Yuan)
Global Internet of Things penetration changes
Consumer Iot: Still in its infancy
Consumer IOT is expected to grow rapidly.
(1) Global: Global consumer IOT hardware sales reached 485.9 billion US dollars in 2017, up 29.5% year on year, with a compound growth rate of 26.0% from 2015 to 2017. The sales volume is expected to reach us $1,550.2 billion in 2022, with a compound annual growth rate of 26.1% from 2017 to 2022. The scale of the global consumer IOT market is further accelerating.
(2) Mainland China: The sales volume of consumer IOT hardware in Mainland China reached $118.8 billion in 2017, with a year-on-year growth of 30.0% and a compound growth rate of 28.9% from 2015 to 2017. Sales are expected to reach us $311.8 billion in 2022, with a compound annual growth rate of 21.3% from 2017 to 2022. Before 2017, due to the rapid development of Xiaomi and other companies, the overall development of China’s consumer IOT is faster than the global average. After 2017, the global consumer IOT will develop faster when China’s consumer IOT still maintains its rapid development. (3) Connected devices: The number of global consumer IOT terminals reached 4.9 billion in 2017, with a compound annual growth rate of 27.7% from 2015 to 2017, and is expected to reach 15.3 billion in 2022, with a compound annual growth rate of 25.4% from 2017 to 2022. In 2017, the number of Consumer IOT terminals in China accounted for 26.5% of the world, which is expected to rise to 29.4% in 2022, and the compound growth rate from 2017 to 2022 is expected to reach 28.2%.
Global consumer IOT market size:China’s consumer IOT market size:Number of IOT terminals worldwide and in China:
200 million controllable demand analysis
2.1 Requirements overview
As a medium station, Yicontrollable can carry out real-time online monitoring and early warning of equipment running status, and does not do business related functions.
List of core features:
(1) Packet data collection and index analysis: The data source of the whole system is the packet message sent by the receiving device. The index data of the subject and message content fields are defined as filtering conditions in the system, so as to collect and analyze messages.
(2) Alarm monitoring: by comparing with data of various alarm levels defined in the system, once a message triggered to the alarm level is found, the alarm information will be transparently transmitted to other systems through webhook associated with the configuration of the alarm
(3) GPS positioning monitoring: Collect THE GPS positioning of each device and provide device position query function.
(4) Data Kanban: Provide rich custom data kanban.
2.2 Service Architecture Diagram
As can be seen from the figure above, the whole system is divided into 6 functional modules: graphics monitoring module, data detail display module, kanban management module, equipment management module, alarm management module and system management module.
2.3 Core Service Description
Product prototype address:
App. Mockplus. Cn/run/prototy…
For details, see “Billion Controllable PRD Document” provided by resources.
300 million controllable system architecture
3.1 System architecture diagram
The technical architecture of the whole system is as follows:
Pre-prepared data is stored in MySQL, indicator data reported by devices including alarm data is stored in influxDB, and device location information is stored in ES for later search. In order to improve the operating stability of the system, some frequently accessed data are stored in Redis. Considering that the data reported by the device is very frequent, if only relying on MySQL database, the CPU usage of MySQL server will easily reach 100%, which will cause the crash of the whole system.
Some basic configurations are put into Consul’s configuration center. Considering the horizontal expansion capability of the system, the entire system is built into a microservice based on Consul as a registry.
3.2 Database Design
Mysql database has 5 tables:
Administrator table tb_admin
The column name | The data type | instructions |
---|---|---|
id | int | Id of the primary key of the table |
login_name | varchar(50) | Login account |
password | varchar(60) | password |
type | tinyint | Type 1: super administrator 2: Common User This field is reserved |
board | varchar(50) | The kanban list |
Kpi configuration table TB_quota
The column name | The data type | instructions |
---|---|---|
id | int | Table primary key id |
name | varchar(50) | The index name |
unit | varchar(20) | Index of the unit |
subject | varchar(50) | Message theme |
value_key | varchar(50) | Index value field |
sn_key | varchar(50) | Device identifier field |
webhook | varchar(1000) | Web hooks |
value_type | varchar(10) | The value can be Double, Inteter, or Boolean |
reference_value | varchar(100) | reference |
Alarm configuration table TB_alarm
The column name | The data type | instructions |
---|---|---|
id | int | Id of the primary key of the table |
name | varchar(50) | Alarm indicator name |
quota_id | int | Name of Associated Indicator |
operator | varchar(10) | The operator |
threshold | int | The alarm threshold |
level | int | Alarm level 1: minor 2: major |
cycle | int | Silence period (in minutes) |
webhook | varchar(1000) | Web Hook address |
Panel configuration table TB_board
The column name | The data type | instructions |
---|---|---|
id | int | Id of the primary key of the table |
admin_id | int | Administrator id |
name | varchar(50) | The name of the kanban |
quota | varchar(100) | indicators |
device | varchar(100) | equipment |
system | tinyint | Is it system Kanban |
disable | tinyint | Whether not to display |
GPS configuration table TB_GPS
The column name | The data type | instructions |
---|---|---|
id | bigint | Table primary key id |
subject | varchar(50) | Message theme |
sn_key | varchar(50) | Device identifier field |
type | tinyint | Type (single field, double field) |
value_key | varchar(50) | Latitude and longitude field |
separation | varchar(10) | Latitude and longitude delimiter |
longitude | varchar(20) | Longitude field |
latitude | varchar(20) | Dimension field |
4. Basic code analysis
4.1 Environment Preparations
4.1.1 Loading a VM Image
Use the virtual machine image that comes with the course.
NAT mode is recommended for network connection.
The code provided in the course notes, 192.168.200.128, is the host IP address, if you load the image after the IP address is not correct, please adjust.
Docker environment has been installed and the required image has been pulled out of the box.
4.1.2 MySQL Database Construction Table construction
The username root and password root123 are used to connect to the mysql database of the VM
Create database yKk, create table
create table if not exists tb_admin
(
id int auto_increment
primary key,
login_name varchar(50) null comment 'Login name',
password varchar(60) null comment 'password',
type tinyint null comment 'Type 1 Super Administrator 0 Common User',
board varchar(50) null comment 'kanban'
);
create table if not exists tb_alarm
(
id int auto_increment comment 'id'
primary key,
name varchar(50) null comment 'Alarm Name',
quota_id int null comment 'index id',
operator varchar(10) null comment 'operator',
threshold int null comment 'Alarm threshold',
level int null comment 'Alarm level 1 Moderate 2 Severe'.cycle int null comment 'Silence period (min)',
webhook varchar(1000) null comment 'web hooks'.constraint tb_alarm_name_uindex
unique (name)
);
create table if not exists tb_board
(
id int auto_increment comment 'id'
primary key,
admin_id int default 1 null comment 'Administrator ID',
name varchar(50) null comment 'Kanban Name',
quota varchar(100) default '0' null comment 'Indicator (set when trending)',
device varchar(100) null comment 'Equipment (Accumulated)',
`system` tinyint default 0 null comment 'Is it system Kanban?',
disable tinyint default 0 null comment 'Do I not show?'.constraint tb_board_name_uindex
unique (name)
);
create table if not exists tb_gps
(
id int not null comment 'id'
primary key,
subject varchar(50) null comment 'theme',
sn_key varchar(50) null comment 'Device Identification Field',
single_field tinyint null comment 'Type (single field, double field)',
value_key varchar(50) null comment 'Latitude and longitude field',
separation varchar(10) null comment 'Latitude and longitude separator',
longitude varchar(20) null comment 'Longitude field',
latitude varchar(20) null comment 'Dimension field'.constraint tb_gps_subject_uindex
unique (subject)
);
create table if not exists tb_quota
(
id int auto_increment comment 'id'
primary key,
name varchar(50) null comment 'Indicator name',
unit varchar(20) null comment 'Indicator unit',
subject varchar(50) null comment 'Message subject',
value_key varchar(50) null comment 'Index value field',
sn_key varchar(50) null comment 'Device Identification Field',
webhook varchar(1000) null comment 'web hooks',
value_type varchar(10) null comment 'Index field type, Double, Inteter, Boolean',
reference_value varchar(100) null comment 'Reference value'.constraint tb_quota_name_uindex
unique (name)
);
Copy the code
4.1.3 Consul Add configuration
(1) Enter Consul
Open the browser, enter the address http://192.168.200.128:8500/
(2) Set the key to config/backend-service/data value as follows
spring:
datasource:
url: JDBC: mysql: / / 192.168.200.128:3306 / ykk? useUnicode=true&autoReconnect=true&autoReconnectForPools=true&characterEncoding=utf8&serverTimezone=Asia/Shanghai
username: root
password: root123
driver-class-name: com.mysql.jdbc.Driver
redis:
host: 192.168200.128.
port: 6379
database: 0
lettuce:
pool:
max-active: 10
max-wait: - 1
max-idle: 5
min-idle: 1
shutdown-timeout: 100
timeout: 1000
password:
Copy the code
4.2 Engineering structure analysis
The screenshot of the main frame of the project is as follows:
Currently, the project is divided into two parts: YKK-Common and YKK-Backend.
The YKK-common module holds some basic commonality definitions of the system, including common exception definitions, database join definitions, and some constant definitions.
The ykk-Backend module is the implementation code of our backend logic, which is divided into specific packages based on specific functions.
4.3 Core code analysis
4.3.1 User login and JWT verification
(1) User login service logic
package com.yikekong.service.impl;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import com.google.common.base.Strings;
import com.yikekong.entity.AdminEntity;
import com.yikekong.mapper.AdminMapper;
import com.yikekong.service.AdminService;
import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder;
import org.springframework.stereotype.Service;
@Service
public class AdminServiceImpl extends ServiceImpl<AdminMapper.AdminEntity> implements AdminService{
@Override
public Integer login(String loginName, String password) {
if(Strings.isNullOrEmpty(loginName) || Strings.isNullOrEmpty(password)){
return -1;
}
QueryWrapper<AdminEntity> queryWrapper = new QueryWrapper<>();
queryWrapper
.lambda()
.eq(AdminEntity::getLoginName,loginName);
AdminEntity adminEntity = this.getOne(queryWrapper);
if(adminEntity == null)
return -1;
BCryptPasswordEncoder passwordEncoder = new BCryptPasswordEncoder();
if(passwordEncoder.matches(password,adminEntity.getPassword())){
return adminEntity.getId();
}
return -1; }}Copy the code
(2) The user logs in to the controller
@RestController
public class AdminController{
@Autowired
private AdminService adminService;
@PostMapping("/login")
public LoginResultVO login(@RequestBody AdminVO admin){
LoginResultVO result = new LoginResultVO();
Integer adminId = adminService.login(admin.getLoginName(),admin.getPassword());
if(adminId < 0){
result.setLoginSuccess(false);
return result;
}
result.setAdminId(adminId);
String token = JwtUtil.createJWT(adminId);
result.setToken(token);
result.setLoginSuccess(true);
returnresult; }}Copy the code
(3) Login verification
The httpFilter package is used to verify the JWT token, and the AuthFilter class is implemented as follows:
package com.yikekong.httpfilter;
import org.elasticsearch.common.Strings;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Component;
import javax.servlet.*;
import javax.servlet.annotation.WebFilter;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
@Component
@WebFilter(urlPatterns = "/*",filterName = "authFilter")
public class AuthFilter implements Filter{
@Override
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
HttpServletRequest req = (HttpServletRequest)servletRequest;
HttpServletResponse resp = (HttpServletResponse)servletResponse;
String path = ((HttpServletRequest) servletRequest).getServletPath();
// If the login interface is accessed, JWT token verification is not performed
if(path.equals("/login")){
filterChain.doFilter(servletRequest, servletResponse);
return;
}
String authToken = ((HttpServletRequest) servletRequest).getHeader("Authorization");
// How to return verification failure if no Authorization value exists in header
if(Strings.isNullOrEmpty(authToken)){
((HttpServletResponse) servletResponse).setStatus(HttpStatus.UNAUTHORIZED.value());
return;
}
try {
JwtUtil.parseJWT(authToken);
} catch (Exception e) {
// JWT verification failed
((HttpServletResponse) servletResponse).setStatus(HttpStatus.UNAUTHORIZED.value());
return; } filterChain.doFilter(servletRequest, servletResponse); }}Copy the code
4.3.2 Indicator Management – Creating indicators
The create method of QuotaController is used to create metrics
/** * Create index *@param vo
* @return* /
@PostMapping
public boolean create(@RequestBody QuotaVO vo){
QuotaEntity quotaEntity = new QuotaEntity();
BeanUtils.copyProperties(vo,quotaEntity);
return quotaService.save(quotaEntity);
}
Copy the code
The VO class received by this method is the wrapper view object for the front end. There are a lot of times when the data from the front end is not exactly the same as the data from our back-end database, so we usually create a separate VO class for data transfer with the front end. In this way, if the data objects passed by the front end send structural changes, the back-end database structure will not be affected.
BeanUtils.copyProperties(vo,quotaEntity); For copying object data, if two objects have the same property, the property is automatically copied, thus avoiding a large number of setter methods in the code.
5. Device management
5.1 Adding Devices
5.1.1 Requirement analysis
In a billion controllable system, we cannot and do not need to add devices from the system interface. The device is added after the espace EMS receives the packet from the device, parses it, and saves it. Elasticsearch is used as the database of the device because the number of devices used by iot applications can be very large and the data is read and written frequently.
5.1.2 Index library structure design
Equipment library device
The column name | The data type | instructions |
---|---|---|
deviceId | keyword | Equipment serial number |
alarm | boolean | Whether the alarm |
alarmName | keyword | The alarm name |
level | integer | The alarm level |
online | boolean | Whether online |
status | boolean | switch |
tag | keyword | The label |
5.1.3 Code implementation
(1) create indexes library (open kibana create http://192.168.200.128:5601/).
PUT /devices
{
"mappings": {
"properties": {
"deviceId": {
"type": "keyword"
},
"alarm": {
"type": "boolean"
},
"alarmName": {
"type": "keyword"
},
"level": {
"type": "integer"
},
"online": {
"type": "boolean"
},
"status": {
"type": "boolean"
},
"tag": {
"type": "keyword"}}}}Copy the code
(2) Add configuration of POM.xml
<! --es dependencies -->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.7.1</version>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.7.1</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
<version>7.7.1</version>
</dependency>
<! -- End of es dependency -->
Copy the code
(3) Add the configuration to the configuration file. Add the following configuration to the Spring node
elasticsearch:
rest:
uris: http://192.168.200.128:9200
Copy the code
(4) Create package com.yikekong.dto and create DTO class for encapsulating devices
package com.yikekong.dto;
import lombok.Data;
import java.io.Serializable;
/** * Device DTO */
@Data
public class DeviceDTO implements Serializable {
private String deviceId;// Device number
private Boolean alarm;// Whether an alarm is generated
private String alarmName;// Alarm name
private Integer level;// Alarm severity
private Boolean online;// Whether to be online
private String tag;/ / label
private Boolean status;// Switch status
}
Copy the code
(5) Create com.yikekong.es package, create ESRepository class under the package, and write methods to add devices
package com.yikekong.es;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.yikekong.dto.DeviceDTO;
import com.yikekong.util.JsonUtil;
import lombok.extern.slf4j.Slf4j;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import java.io.IOException;
import java.util.Map;
@Component
@Slf4j
public class ESRepository{
@Autowired
private RestHighLevelClient restHighLevelClient;
/** * Add device *@param deviceDTO
*/
public void addDevices(DeviceDTO deviceDTO){
if(deviceDTO==null ) return;
if(deviceDTO.getDeviceId()==null) return;
IndexRequest request=new IndexRequest("devices");
try {
String json = JsonUtil.serialize(deviceDTO);
Map map = JsonUtil.getByJson(json, Map.class);
request.source(map);
request.id(deviceDTO.getDeviceId());
restHighLevelClient.index(request, RequestOptions.DEFAULT);
} catch (JsonProcessingException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
log.error("Device addition exception occurred"); }}}Copy the code
5.1.4 Unit tests
Writing unit tests
import com.yikekong.YkkApplication;
import com.yikekong.dto.DeviceDTO;
import com.yikekong.es.ESRepository;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;
@SpringBootTest(classes = YkkApplication.class)
@RunWith(SpringRunner.class)
public class EsTest {
@Autowired
private ESRepository esRepository;
@Test
public void testAdd(a){
DeviceDTO deviceDTO=new DeviceDTO();
deviceDTO.setDeviceId("123456");
deviceDTO.setStatus(true);
deviceDTO.setAlarm(false);
deviceDTO.setLevel(0);
deviceDTO.setAlarmName("");
deviceDTO.setOnline(true);
deviceDTO.setTag(""); esRepository.addDevices(deviceDTO); }}Copy the code
Query data and verify running results
GET devices/_search
{
"query": {
"match_all": {}}}Copy the code
5.2 Querying Devices By Device ID
5.2.1 Requirement analysis
Query device information from ElasticSearch by ID. You need to invoke this method in the packet parsing logic to query the device.
5.2.2 Code implementation
ESRepository class add method
/** * Query devices based on device ID *@paramDeviceId deviceId *@return* /
public DeviceDTO searchDeviceById(String deviceId){
SearchRequest searchRequest=new SearchRequest("devices");
SearchSourceBuilder searchSourceBuilder=new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.termQuery("_id",deviceId));
searchRequest.source(searchSourceBuilder);
try {
SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
long hitsCount = hits.getTotalHits().value;
if(hitsCount<=0) return null;
DeviceDTO deviceDTO=null;
for(SearchHit hit:hits){
String hitResult = hit.getSourceAsString();
deviceDTO=JsonUtil.getByJson(hitResult,DeviceDTO.class );
deviceDTO.setDeviceId(deviceId);
break;
}
return deviceDTO;
} catch (IOException e) {
e.printStackTrace();
log.error("Abnormal device query");
return null; }}Copy the code
5.2.3 Unit testing
Write unit tests to verify that the code is correct
@Test
public void testSearchById(a){
DeviceDTO deviceDTO = esRepository.searchDeviceById("123456");
try {
String json = JsonUtil.serialize(deviceDTO);
System.out.println(json);
} catch(JsonProcessingException e) { e.printStackTrace(); }}Copy the code
5.3 Setting the Device Status
5.3.1 Requirements analysis
If you do not need to receive packets from a device, you can disable it. The disabled device does not process the indicators of received packets.
5.3.2 API interface
5.3.3 Code implementation
(1) Add methods of ESRepository class to enable and disable devices.
/** * Update device status *@param deviceId
* @param status
* @return* /
public boolean updateStatus(String deviceId,Boolean status){
UpdateRequest updateRequest=new UpdateRequest("devices",deviceId)
.doc( "status",status );
try {
restHighLevelClient.update( updateRequest,RequestOptions.DEFAULT );
return true;
} catch (IOException e) {
e.printStackTrace();
log.error("Error updating device status");
return false; }}Copy the code
(2) DeviceService added method definition
/** * Change the device status *@param deviceId
* @param status
* @return* /
boolean setStatus(String deviceId, Boolean status);
Copy the code
DeviceServiceImpl implementation method
@Autowired
private ESRepository esRepository;
@Override
public boolean setStatus(String deviceId, Boolean status) {
DeviceDTO deviceDTO = findDevice(deviceId);
if( deviceDTO==null ) return false;
return esRepository.updateStatus(deviceId,status);
}
/** * Query devices based on device ID *@param deviceId
* @return* /
private DeviceDTO findDevice(String deviceId){
DeviceDTO deviceDTO = esRepository.searchDeviceById(deviceId);
return deviceDTO;
}
Copy the code
(3) DeviceController added method
/** * Sets the status of the interface *@param deviceVO
* @return* /
@PutMapping("/status")
public boolean setStatus(@RequestBody DeviceVO deviceVO){
return deviceService.setStatus(deviceVO.getSn(),deviceVO.getStatus());
}
Copy the code
5.3.4 Interface Test
(1) Run YkkApplication to start the project
(2) Test interface. To facilitate testing, we used vscode’s Rest Client plug-in for testing. The script is provided in the course “data \ test \yikekong. HTTP”. Open it with vscode and find the following script
#### Changing the device status ######## PUT HTTP:/ / {{the hostname}}, {{port}} / device/status of HTTP / 1.1
Authorization: {{Authorization}}
Content-Type: {{contentType}}
{
"sn":"123456"."status":true
}
Copy the code
We can test by changing the status value and clicking the Send Request link
5.4 Setting Device Labels
5.4.1 Requirement analysis
We can set one or more labels for each device in order to facilitate the query of the device later. There is no device label update function on the front-end interface. This function only provides an interface to call external systems.
5.4.2 Code implementation
(1) Add method of ESRepository class
/** * Update the device label *@param deviceId
* @param tags
* @return* /
public boolean updateDeviceTag(String deviceId,String tags){
UpdateRequest updateRequest=new UpdateRequest("devices",deviceId)
.doc( "tag",tags );
try {
restHighLevelClient.update( updateRequest,RequestOptions.DEFAULT );
return true;
} catch (IOException e) {
e.printStackTrace();
log.error("Error updating device label");
return false; }}Copy the code
(2) Adding method of DeviceService
/** * Update the device label *@param deviceId
* @param tags
* @return* /
boolean updateTags(String deviceId,String tags);
Copy the code
DeviceServiceImpl implementation method
@Override
public boolean updateTags(String deviceId, String tags) {
DeviceDTO deviceStatus = findDevice(deviceId);
if(deviceStatus == null) return false;
esRepository.updateDeviceTag(deviceId,tags);
return true;
}
Copy the code
(3) DeviceController added method
/** * Sets the label interface *@param deviceVO
* @return* /
@PutMapping("/tags")
public boolean setTags(@RequestBody DeviceVO deviceVO){
return deviceService.updateTags(deviceVO.getSn(),deviceVO.getTags());
}
Copy the code
(4) doFilter new code of AuthFilter class, allowing tags
//tag The interface does not verify the token
if(path.contains("/device/tags")){
filterChain.doFilter(servletRequest, servletResponse);
return;
}
Copy the code
5.4.3 Interface Test
(1) Start the project
(2) Find the following script and test it
#### Set device labels ############ PUT HTTP:/ / {{the hostname}}, {{port}} / device/tags HTTP / 1.1
Content-Type: {{contentType}}
{
"sn":"123456"."tags":"School"
}
Copy the code
5.5 Updating Device Alarm Information
5.5.1 Requirement analysis
If the indicator information in the packet sent by the device reaches the alarm level, update the device alarm information (alarm, alarm severity, and alarm name) in ElasticSearch.
5.5.2 Code implementation
ESRepository class add method
/** * Updates device alarm information *@param deviceDTO
* @return* /
public boolean updateDevicesAlarm(DeviceDTO deviceDTO){
UpdateRequest updateRequest=new UpdateRequest("devices",deviceDTO.getDeviceId())
.doc( "alarm",deviceDTO.getAlarm(),// Whether an alarm is generated
"level",deviceDTO.getLevel(),// Alarm severity
"alarmName",deviceDTO.getAlarmName() );// Alarm name
try {
restHighLevelClient.update( updateRequest,RequestOptions.DEFAULT );
return true;
} catch (IOException e) {
e.printStackTrace();
log.error("Error updating device alarm Information");
return false; }}Copy the code
5.5.3 Unit testing
Write unit tests and add test methods to TestES
@Test
public void testAlarm(a){
DeviceDTO deviceDTO=new DeviceDTO();
deviceDTO.setDeviceId("123456");
deviceDTO.setAlarm(true);
deviceDTO.setLevel(1);
deviceDTO.setAlarmName("Excessive temperature");
esRepository.updateDevicesAlarm(deviceDTO);
}
Copy the code
5.6 Updating the Online status
5.6.1 Requirement analysis
Online status indicates whether the device is online. If a network fault occurs on the device, the device is offline. The billion controllable system can monitor the online and offline status of the equipment
5.6.2 Code implementation
Here we need to add methods to the ESRepository class to update the online status.
/** * Update online status *@param deviceId
* @param online
* @return* /
public boolean updateOnline(String deviceId,Boolean online){
UpdateRequest updateRequest=new UpdateRequest("devices",deviceId)
.doc( "online",online );
try {
restHighLevelClient.update( updateRequest,RequestOptions.DEFAULT );
return true;
} catch (IOException e) {
e.printStackTrace();
log.error("Error updating online status");
return false; }}Copy the code
5.6.3 Unit tests
Write unit tests and add test methods to TestES
@Test
public void testOnline(a){
esRepository.updateOnline("123456".false);
}
Copy the code
5.7 Querying Device Information on multiple Pages
5.7.1 Requirement analysis
There are two pages that need to implement the paging query device
(1) Device management: as shown in the following figure, device number and label should be used as query conditions for paging query
(2) Device details, as shown in the following figure, need device status, label and device number as query conditions
The device details page provides a query condition for Device status. The device status has four values: Online (0), Offline (1), Minor alarm (2), and Major Alarm (3).
To avoid code redundancy, we can implement both functions in the same way.
5.7.2 Code implementation
ESRepository class add method
/** * page query device *@paramPage page *@paramPageSize pageSize *@paramDeviceId deviceId *@paramTags, labels *@paramState of the state *@return* /
public Pager<DeviceDTO> searchDevice(Long page,Long pageSize,String deviceId,String tags,Integer state){
SearchRequest searchRequest=new SearchRequest("devices");
SearchSourceBuilder sourceBuilder=new SearchSourceBuilder();
// Conditional query
BoolQueryBuilder boolQueryBuilder=QueryBuilders.boolQuery();
// Device number
if(! Strings.isNullOrEmpty(deviceId)) { boolQueryBuilder.must(QueryBuilders.wildcardQuery("deviceId", deviceId + "*"));
}
/ / label
if(! Strings.isNullOrEmpty(tags) ){ boolQueryBuilder.must(QueryBuilders.wildcardQuery("tag"."*"+tags+"*"));
}
// Status (online and alarm) 0: online 1: offline 2: minor 3: major
if(state! =null) {if(state.intValue()==0){
boolQueryBuilder.must( QueryBuilders.termQuery("online".true));
}
if(state.intValue()==1){
boolQueryBuilder.must( QueryBuilders.termQuery("online".false));
}
if(state.intValue()==2){
boolQueryBuilder.must( QueryBuilders.termQuery("level".1));
}
if(state.intValue()==3){
boolQueryBuilder.must( QueryBuilders.termQuery("level".2));
}
}
sourceBuilder.query(boolQueryBuilder);
/ / paging
sourceBuilder.from( (page.intValue()-1)*pageSize.intValue() );
sourceBuilder.size( pageSize.intValue() );
sourceBuilder.trackTotalHits(true);
/ / sorting
sourceBuilder.sort("level", SortOrder.DESC);// The alarm severity is higher than the alarm severity
searchRequest.source(sourceBuilder);
try {
SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
SearchHit[] searchHits = hits.getHits();
List<DeviceDTO> devices= Lists.newArrayList();
for(SearchHit hit: searchHits){
String hitResult = hit.getSourceAsString();
DeviceDTO deviceDTO = JsonUtil.getByJson(hitResult, DeviceDTO.class);
devices.add(deviceDTO);
}
Pager<DeviceDTO> pager=new Pager<>( searchResponse.getHits().getTotalHits().value,pageSize );
pager.setItems(devices);
return pager;
} catch (IOException e) {
e.printStackTrace();
log.error("Device query failed");
return null; }}Copy the code
(2) DeviceService added method definition
/** * search device *@param page
* @param pageSize
* @param sn
* @param tag
* @return* /
Pager<DeviceDTO> queryPage(Long page, Long pageSize, String sn, String tag, Integer status);
Copy the code
DeviceServiceImpl implementation method
@Override
public Pager<DeviceDTO> queryPage(Long page, Long pageSize, String sn, String tag, Integer status) {
return esRepository.searchDevice(page,pageSize,sn,tag,status);
}
Copy the code
(3) DeviceController added method
/** * paging search device *@param page
* @param pageSize
* @param sn
* @param tag
* @return* /
@GetMapping
public Pager<DeviceDTO> findPage(@RequestParam(value = "page",required = false,defaultValue = "1") Long page,
@RequestParam(value = "pageSize",required = false,defaultValue = "10") Long pageSize,
@RequestParam(value = "sn",required = false) String sn,
@RequestParam(value = "tag",required = false) String tag){
return deviceService.queryPage(page,pageSize,sn,tag,null);
}
Copy the code
V(cmL46679910)