In the previous article, the kettle platform was used to implement the data exchange function between MQTT and Websocket, which can basically solve the communication problems in the Internet of Things platform. This chapter explains another advanced function of the Kettle platform, which is to implement transparent forwarding between Websockets.

Scenario Description:

When I was developing the Internet of Things platform, I ran into this problem. If you want to display real-time device data in the front page, using Websocket communication is an inevitable choice. However, in order to adapt to different types of front-end (H5, APP, applets, etc.), the data format must be different. Of course, the connection with customers also leads to differences in data formats. For example, there are now two companies on the platform, and each company has its own standard protocol. Company A uses the standard JSON format, and Company B uses the standard CXML format (A type of XML, CXML is widely used in SAP). How do you solve this problem in this scenario? The Websocket interface provided by the Internet of Things platform cannot be easily changed and can be considered as an atomic interface. But if you can’t easily change it, how does it fit across platforms?

For this scenario, IT occurred to me that if it was a normal REST interface, instead of a WebSocket interface, you could build a new interface to convert data formats inside the new interface. Because REST interfaces are short-connected, everything is possible. But for long connections like WebSocket, this is obviously not possible. If you can achieve long connection between the transfer can not be solved.

  • demand

Calls between WebSocket interfaces.

  • The problem

1. How do Websocket interfaces communicate with each other? Are multiple WebSocket servers involved?

2. How do interface calls convert data formats to each other?

  • plan

There is not much valuable information through searching the scheme on the Internet, so we are going to implement it in the form of hard coding at the beginning. However, this way will lead to the service not being extensible, and the long connection may be disconnected if new services are added. The ETL feature of Kettle makes it easy to convert data formats. However, calling the WebSocket service of Kettle is not supported. It was decided to write a plug-in to implement this functionality, and eventually to make websoket services call each other.

Effect preview:

Websocket server: Starts the server and pushes the message to the client five times after receiving it.

​

Data integration platform:

The webSocket interface provided by the platform is connected to the WebSocket server.

​

Websocket client online test:

By calling the WebSocket interface, the platform starts to monitor the changes of webSocket server data. If any data enters, it will be pushed out through webSocket.

​

Websocket server console output:

​

Principle Introduction:

  • The WebSocket Client Test node is used to connect to the WebSocket server. This node is implemented by writing the Kettle plug-in.

​

  • Stream processing is performed through the “Get Records from Stream” node, customizing each piece of data. Of course, this is common to MQTT’s flow processing.

​

Combining the above two conversion files, we can realize the monitoring of Websocket service and the processing of stream data.

  • Platform principle:

After the two files are uploaded and saved, the WebSocket service interface uses the two files to associate the webSocket client long connections in the online test with the WebSocket client long connections in the Kettle. Output through the websocket long connection associated channel established by the platform.

Conclusion:

The platform solves the problem of calling websockets by associating long connections between websockets. This completes the chain of webSocket calls. Of course, there are not many application scenarios now, but I believe there will be rich application scenarios in the future.

If have any question, can add QQ893393953.