I was fortunate enough to participate in the development of a smart home project before, and I was curious about the development mode and technology stack of intelligent hardware because I had no previous development experience in this area.

Intelligent combustible gas alarm

The product is a combustible gas alarm. If the concentration of gas leakage at home reaches a certain threshold, the alarm detects and uploads the gas concentration value to the background. The background reminds users of possible gas leakage at home through phone calls, short messages, wechat and other means.

The user may also send some instructions to the alarm to close the alarm and adjust the volume. The overall function is relatively simple, and the general logic is shown in the figure below:

But when I actually participated in the development, I was actually a little disappointed, because in the whole development process, there was no new technology, it was just a few common middleware, just a change of usage.

The technology selection uses RabbitMQ as the core component, mainly considering the low operation and maintenance cost and the high proficiency of the group members.

Here is how to build an Internet of Things (IOT) platform with SpringBoot and RabbitMQ.

A lot of you might be confused, right? Isn’t RabbitMQ a message queue? How can you make smart hardware again?

Rabbitmq has two protocols, AMQP for message queues and MQTT for intelligent hardware.

What is MQTT protocol?

MQTT (Message Queue Telemetry Transport) : A lightweight publish/subscribe protocol that gets messages by subscribing to a topic. It is a standard transport protocol in the Internet of Things.

This protocol connects the publisher of a message (publisher) and subscribers (subscriber), so reliable messaging services can be provided to remotely connected devices in an unreliable network environment, somewhat similar to traditional MQ.

TCP is located at the transport layer, MQTT is located at the application layer, MQTT is built on TCP/IP, that is to say, MQTT can be used wherever TCP/IP stack is supported.

Second, why use MQTT protocol?

Why is MQTT protocol so preferred in the Internet of Things (IOT)? As opposed to some other protocol, like the more familiar HTTP protocol?

  • First of all, HTTP protocol is a synchronization protocol. After the client requests, it needs to wait for the response of the server. In the Internet of Things (IOT) environment, where devices are subject to environmental influences such as low bandwidth, high network latency, and unstable network communication, asynchronous messaging protocols are more suitable for IOT applications.

  • HTTP is one-way and a client must initiate a connection to get a message, whereas in Internet of Things (IOT) applications, devices or sensors tend to be clients, meaning they cannot passively receive commands from the network.

  • Typically, a command or message is sent to all devices on the network. This is not only difficult to do with HTTP, but also extremely expensive.

Iii. Introduction of MQTT protocol

As mentioned earlier, MQTT is a lightweight protocol that focuses only on messaging, so the structure of this protocol is very simple.

MQTT packets

In MQTT protocol, an MQTT packet consists of Fixed header, Variable header, and message body payload.

  • Fixed header: All packets have a Fixed header, which contains the packet type and packet group identifier.
  • Variable headers. Some packet types have Variable headers.
  • Payload, which exists in some data packets, is the specific content of the message received by the client.
Insert a picture description here

1. Fix the head

Fixed header, using two bytes, total 16 bits:The bits (4-7) indicate the message type. The 4-bit binary representation can represent the following 16 message types, but the positions 0 and 15 are reserved for later use, so there are 14 message event types in total.

DUP Flag (Retry Flag)

DUP Flag: indicates the Flag that ensures the reliable transmission of messages and whether messages have been delivered. The default value is 0 and takes only one byte, indicating the first time the message was sent. A value of 1 indicates that the current message has been sent previously.

QoS Level (message quality Level)

QoS Level: indicates the message quality Level, which will be described later

RETAIN (Persistence)

  • A value of 1 indicates that the message sent should be persistent and not affected by server restart. It should not only be sent to current subscribers, but also subscribers will be pushed immediately after new clients subscribe to this Topic. Note: For newly added subscribers, only the latest RETAIN Flag = 1 message push will be retrieved.

  • A value of 0: This message is pushed only for the current subscriber.

Remaining Length

The number of bytes remaining in the current message, including the variable header and message body payload.

2. Variable head

The fixed header defines only the message type and some flag bits, and some of the message metadata needs to be put into the mutable header. Variable header content length + message body payload = Remaining length.

The variable header is between the fixed header and payload. It contains the protocol name, version number, connection flag, user authorization, and heartbeat time.

Variable headers exist in these types of messages: PUBLISH (QoS > 0), PUBACK, PUBREC, PUBREL, PUBCOMP, SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK.

3. Payload

Payload only exists in CONNECT, PUBLISH, SUBSCRIBE, SUBACK, and UNSUBSCRIBE messages.

  • CONNECT: Includes clientsClientIdAnd subscriptionTopic,MessageAs well asThe user nameandpassword.
  • PUBLISH: Sends messages to the corresponding topic.
  • SUBSCRIBE: Topics to subscribe to andQoS.
  • SUBACK: Server forSUBSCRIBEThe subject of application andQoSConfirm and reply.
  • UNSUBSCRIBE: Unsubscribe a topic to subscribe to.

Message Quality (QoS)

Quality of Service is the sending Quality of messages. Publisher and subscriber can specify qos levels, including qos 0, qos 1, and qos 2.

The following illustrates the differences between the three grades.

1, Qos 0

Qos 0: At most once A message is sent only once. Whether the message is successfully delivered is not guaranteed. There is no confirmation mechanism, and the message may be lost or repeated.

Pictures from the network, such as infringement contact delete

2, Qos 1

Qos 1: QoS 1 has an ack acknowledgement mechanism compared with QoS 0, and both publishers persist messages when they push messages to MQTT brokers. The publisher or Broker will delete its own persistent message only if it receives a PUBACK acknowledgement, or it will resend it.

But there’s a problem. While we can make sure that we get a message from a client or server by acknowledging it, we can’t make sure that we get a message only once. That is, when the client publisher does not receive the Broker’s puback or the Broker does not receive the subscriber’s puback, it will be reissued all the time.

Publisher -> Broker

  1. Publisher store MSG -> publish ->broker
  2. Broker -> puback -> publisher delete MSG
Pictures from the network, such as infringement contact delete

3, Qos 2

Qos 2: Publisher and Broker also persist messages, where Publisher caches messages and their corresponding MSgIds. The broker caches msGids to ensure that messages are not duplicated, but the process is complicated by the addition of a confirm mechanism.

Publisher -> Broker

  1. publisher store msg -> publish ->broker -> broker store
  2. MsgID (Delivering message) Broker -> puberc (Verifying delivery success)
  3. Publisher -> pubrel ->broker delete msgID
  4. Broker -> pubcomp -> publisher delete MSG
Pictures from the network, such as infringement contact delete

LWT (Last Will)

LWT is called Last Will and Testament. In fact, Will is a subject and corresponding message pre-defined by the client, which is attached to the packet of CONNECT, including the subject, QoS and message of the Will.

When the MQTT Broker detects that a client is abnormally disconnected, the server proactively publishes the message, and relevant subscribers receive the message.

For example, everyone in the chat room subscribed to a topic called Talk, but rich suddenly disconnects due to network jitters. All clients subscribed to the topic talk in the chat room will receive a wish message “Rich left the chat room”.

Related parameters of the will:

  • Will Flag: Whether to use LWT, 1 is enabled
  • Will Topic: Will subject name. Wildcard characters cannot be used
  • Will Qos: QoS used when publishing will messages
  • Will Retain: Retain identifier for the will message
  • Will Message: Will message content

The clientClientWhat are the scenarios of abnormal disconnection?

  • BrokerAn underlying I/O exception is detected.
  • Client failed in heartbeatKeep AliveThe interval sum ofBrokerConduct message interaction;
  • The client is closing the bottom layerTCPNo sending before connectionDISCONNECTPackets;
  • The client sent a packet with an incorrect format. ProcedureBrokerTo close the connection to the client.

Note: When the client disconnects by publishing the DISCONNECT packet, it is a normal disconnection and does not trigger LWT. At the same time, the Broker will also discard the relevant LWT parameters specified by the current client when connecting.

Iv. Application scenarios of MQTT protocol

MQTT protocol is widely used in Internet of Things, mobile Internet, intelligent hardware, Internet of vehicles, power energy and other fields. There are many different scenarios, here are some:

  • Internet of Things M2M communication, Internet of Things big data collection
  • Android notification push, WEB notification push
  • Mobile instant messaging, such as Facebook Messenger
  • Intelligent hardware, intelligent furniture, intelligent appliances
  • Internet of vehicles communication, electric station pile acquisition
  • Smart city, telemedicine, distance education
  • Power, oil and energy industries

Five, code implementation

Rabbitmq environment setup will not be described, more online tutorials, conditional use of the server, not conditional like I do a Windows version is also very happy.

Insert a picture description here

1. Enable the MQTT protocol for RabbitMQ

To enable the MQTT protocol for RabbitMQ, which is disabled by default, run the following command:

rabbitmq-plugins enable rabbitmq_mqtt
Copy the code

2. MQTT client dependency package

After installing the RabbitMQ environment and enabling the MQTT protocol in the previous step, the MQTT message broker service is now set up and the next step is to implement client push and subscribe messages.

Using spring here – integration – the MQTT, org. Eclipse paho. Client. Mqttv3 two kit.

<! -- MQTT dependencies --><dependency>
    <groupId>org.springframework.integration</groupId>
    <artifactId>spring-integration-mqtt</artifactId>
</dependency>
<dependency>  <groupId>org.eclipse.paho</groupId>  <artifactId>org.eclipse.paho.client.mqttv3</artifactId>  <version>1.2.0</version> </dependency> Copy the code

3. Message sender

Message sending is simple, mainly applied to the @ServiceActivator annotation. Note that the MessageHandler. setAsync property, if set to false, may block when sending messages in asynchronous mode.

@Configuration
public class IotMqttProducerConfig {

    @Autowired
    private MqttConfig mqttConfig;
  @Bean  public MqttPahoClientFactory mqttClientFactory() {  DefaultMqttPahoClientFactory factory = new DefaultMqttPahoClientFactory();  factory.setServerURIs(mqttConfig.getServers());  return factory;  }   @Bean  public MessageChannel mqttOutboundChannel() {  return new DirectChannel();  }   @Bean  @ServiceActivator(inputChannel = "iotMqttInputChannel")  public MessageHandler mqttOutbound() {  MqttPahoMessageHandler messageHandler = new MqttPahoMessageHandler(mqttConfig.getServerClientId(), mqttClientFactory());  messageHandler.setAsync(false);  messageHandler.setDefaultTopic(mqttConfig.getDefaultTopic());  return messageHandler;  } } Copy the code

When MQTT provides an API for sending messages, it needs to use the @MessagingGateway annotation to provide a message gateway proxy, with defaultRequestChannel specifying the channel to send the message binding.

Payload is the sent message. Topic is the topic of the sent message. Qos message quality.

@MessagingGateway(defaultRequestChannel = "iotMqttInputChannel")
public interface IotMqttGateway {

    // Send messages to the default topic
    void sendMessage2Mqtt(String payload);
 // Send messages to the specified topic  void sendMessage2Mqtt(String payload,@Header(MqttHeaders.TOPIC) String topic);  // Send messages to the specified topic and specify quality of service parameters  void sendMessage2Mqtt(@Header(MqttHeaders.TOPIC) String topic, @Header(MqttHeaders.QOS) int qos, String payload); }  Copy the code

4. Message subscription

Message subscription is similar to our usual MQ message listening implementation, with the @ServiceActivator annotation indicating that the current method is used to process MQTT messages and the inputChannel parameter specifying the channel used to receive messages.

/ * * * @Author: xiaofu
* @description: message subscription configuration * @date 2020/6/8 18:24
* /
@Configuration public class IotMqttSubscriberConfig {   @Autowired  private MqttConfig mqttConfig;   @Bean  public MqttPahoClientFactory mqttClientFactory() {  DefaultMqttPahoClientFactory factory = new DefaultMqttPahoClientFactory();  factory.setServerURIs(mqttConfig.getServers());  return factory;  }   @Bean  public MessageChannel iotMqttInputChannel() {  return new DirectChannel();  }   @Bean  public MessageProducer inbound() {  MqttPahoMessageDrivenChannelAdapter adapter = new MqttPahoMessageDrivenChannelAdapter(mqttConfig.getClientId(), mqttClientFactory(), mqttConfig.getDefaultTopic());  adapter.setCompletionTimeout(5000);  adapter.setConverter(new DefaultPahoMessageConverter());  adapter.setQos(1);  adapter.setOutputChannel(iotMqttInputChannel());  return adapter;  }   / * * * @author xiaofu * @description message subscription * @date 2020/6/8 18:20 * /  @Bean  @ServiceActivator(inputChannel = "iotMqttInputChannel")  public MessageHandler handlerTest() {   return message -> {  try {  String string = message.getPayload().toString();  System.out.println("Received message:" + string);  } catch (MessagingException ex) {  //logger.info(ex.getMessage());  }  };  } } Copy the code

Test message

Um ~ because this slag is ignorant of hardware, in order to simulate the hardware to send a message, can only use the tools, in fact, the hardware end to achieve MQTT protocol, with our front of the basic no difference, but in a language embedded in the hardware.

Choose test tool for mqttbox here, the download address: http://workswithweb.com/mqttbox.html

1. Test message sending

We simulate sending a message to the topic MQTT_test_topic with mqTTBox first to see if the background receives it successfully.

See backstage successfully get to the thememqtt_test_topicSent message.

2. Test message subscription

Use mqTTBox to simulate the subscription topic MQTT_test_topic and send a message to the topic MQTT_test_topic in the background. Here I simply write a Controller call API to send the message.

http://127.0.0.1:8080/fun/testMqtt?topic=mqtt_test_topic&message= I send messages to the theme mqtt_test_topic is the backgroundWe look atmqttboxSubscribe message, has successfully received the background message, to ourMQTTThe communication environment is set up successfully. If themqttboxThe tool changes specific hardware equipment, whole flow is the intelligent household that we often say namely, did not have so difficult really actually.

Seven, application matters needing attention

In our actual production environment encountered problems, here to share to let you less pit.

ClientId is the only

When clients connect, there is a clientId parameter that needs to be unique for each client. However, clientId died directly in the code during our development and testing phase, and the services were single-instance deployment, which didn’t expose any problems.

MqttPahoMessageDrivenChannelAdapter(mqttConfig.getClientId(), mqttClientFactory(), mqttConfig.getDefaultTopic());
Copy the code

However, in a production environment, the following strange problem occurs because the service is clustered with multiple instances. Only one client can receive messages at a time. Other clients cannot consume messages and continue to lose connections. retrying… .

And that’s becauseclientIdThe same causes clients to compete with each other for consumption, and eventually willclientIdThe acquisition method is changed to take from the transmitter, the problem is fine, so this area is to need special attention.

Normally the application is fine in the development environment, but in the production environment, there are a lot of problems, many of which are caused by the way the service is deployed. So it is necessary to learn more about distribution.

Other middleware

MQTT is just a protocol, and there are many messaging middleware products that support MQTT, of which the following are just a few

  • Mosquitto
  • Eclipse Paho
  • RabbitMQ
  • Apache ActiveMQ
  • HiveMQ
  • JoramMQ
  • ThingMQ
  • VerneMQ
  • Apache Apollo
  • emqttd Xively
  • IBM Websphere …..

conclusion

It is also the first time for me to do a hardware-related project. When I heard about smart home, I always thought it was great. However, after the actual development, I found that the technology has changed and is just used in a different way.

Hands on demo project making address: https://github.com/chengxy-nds/springboot-rabbitmq-mqtt.git, interested friends can download run a run, very simple to implement.

Original is not easy, burning hair output content, I hope you can have a drop drop harvest!

I sorted out hundreds of technical e-books and gave them to my friends. Close the public account reply [666] to collect by yourself. I set up a technology exchange group with some friends to discuss technology and share technical information, aiming to learn and progress together. If you are interested, just scan code and join us!