preface
Recently, I was working on a small tool. There was a requirement to view log files in real time on the Web side, i.e. execute tail -f command on the terminal. I couldn’t find a good solution to this problem.
public static void main(String[] args) throws Exception {
File file = new File("/home/1.txt");
FileInputStream fin = new FileInputStream(file);
int ch;
fin.skip(10);
while((ch = fin.read()) ! = -1){
System.out.print((char) ch); }}Copy the codeIf we don’t skip it, then it’s not realistic to read everything and display it at a time. Instead, we would like tail to read from the last n lines at a time, and output the latest line continuously.
Another problem is to be aware of the changes in the file, so we choose to call tail directly and output to the web page through the WebSocket.
Tail usage
If you can’t read data from readLine(), it blocks and does not return null. This means that no new data has been written to the log file. Once the readLine() method returns, new data has arrived. The other question is how to terminate. We can’t make it read all the time. We need to terminate at a good time
public static void main(String[] args) throws Exception {
Process exec = Runtime.getRuntime().exec(new String[]{"bash"."-c"."tail -F /home/HouXinLin/test.txt"});
InputStream inputStream = exec.getInputStream();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
for(;;) { System.out.println(bufferedReader.readLine()+"\r"); }}Copy the codeThe implementation process
There are many ways to add WebSocket functionality to Spring Boot. The most common articles are ServerEndpointExporter, @onOpen, @onclose, @onMessage. This method requires the declaration of a Bean, ServerEndpointExporter, but I remember that if you want to package the Bean as a war and run it in Tomcat, you also need to cancel the Bean, otherwise you will get an error, which is very troublesome, of course there are ways to resolve this.
And other integration solution, such as implementation WebSocketConfigurer or WebSocketMessageBrokerConfigurer interface, and I am currently using is to realize the WebSocketMessageBrokerConfigurer interface, And the front end also requires two libraries, SockJS and Stomp (optionally, or not).
SockJS provides webSocket-like objects and a cross-browser API that creates a low-latency, full-duplex, cross-domain communication channel between the browser and the Web server. If the browser does not support WebSocket, it can also simulate WebSocket support.
Stomp stands for Simple Text Orientated Messaging Protocol. It provides an interoperable connection format that allows Stomp clients to interact with any Stomp message Broker.
Let’s first look at the logic of the connection processing layer, where some of the unnecessary code is not shown.
@Configuration
@EnableWebSocketMessageBroker
public class WebSocketConfig implements WebSocketMessageBrokerConfigurer {
private static final Logger log = LoggerFactory.getLogger(WebSocketConfig.class.getName());
@Autowired
SimpMessagingTemplate mSimpMessagingTemplate;
@Autowired
WebSocketManager mWebSocketManager;
@Autowired
TailLog mTailLog;
@Override
public void configureMessageBroker(MessageBrokerRegistry registry) {
registry.enableSimpleBroker("/topic/path");
}
@Override
public void configureWebSocketTransport(WebSocketTransportRegistration registration) {
registration.addDecoratorFactory(new WebSocketHandlerDecoratorFactory() {
@Override
public WebSocketHandler decorate(WebSocketHandler webSocketHandler) {
return new WebSocketHandlerDecorator(webSocketHandler) {
@Override
public void afterConnectionEstablished(WebSocketSession session) throws Exception {
log.info("Log monitor WebSocket connection,sessionId={}", session.getId());
mWebSocketManager.add(session);
super.afterConnectionEstablished(session);
}
@Override
public void afterConnectionClosed(WebSocketSession session, CloseStatus closeStatus) throws Exception {
mWebSocketManager.remove(session.getId());
super.afterConnectionClosed(session, closeStatus); }}; }}); }@Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
registry.addEndpoint("/socket-log")
.addInterceptors(new HttpHandshakeInterceptor())
.setHandshakeHandler(new DefaultHandshakeHandler() {
@Override
protected Principal determineUser(ServerHttpRequest request, WebSocketHandler wsHandler, Map<String, Object> attributes) {
return new StompPrincipal(UUID.randomUUID().toString());
}
})
.withSockJS();
}
@EventListener
public void handlerSessionCloseEvent(SessionDisconnectEvent sessionDisconnectEvent) {
StompHeaderAccessor headerAccessor = StompHeaderAccessor.wrap(sessionDisconnectEvent.getMessage());
mTailLog.stopMonitor(headerAccessor.getSessionId());
}
/** * path subscription **@param sessionSubscribeEvent
*/
@EventListener
public void handlerSessionSubscribeEvent(SessionSubscribeEvent sessionSubscribeEvent) {
StompHeaderAccessor headerAccessor = StompHeaderAccessor.wrap(sessionSubscribeEvent.getMessage());
if (mTailLog.isArriveMaxLog()) {
mWebSocketManager.sendMessage(headerAccessor.getSessionId(), "Number of monitors reached limit, cannot view \"");
log.info("Log monitor WebSocket connections reached maximum number, will be disconnected sessionId={}", headerAccessor.getSessionId());
mWebSocketManager.close(headerAccessor.getSessionId());
return;
}
String destination = headerAccessor.getDestination();
String userId = headerAccessor.getUser().getName();
if (destination.startsWith("/user/topic/path")) {
String path = destination.substring("/user/topic/path".length());
File file = new File(StringUtils.urlDecoder(path));
if(! file.exists()) { mWebSocketManager.sendMessage(headerAccessor.getSessionId(),"What are you doing? I can't find the file.");
mWebSocketManager.close(headerAccessor.getSessionId());
return;
}
TailLogListenerImpl tailLogListener = new TailLogListenerImpl(mSimpMessagingTemplate, userId);
mTailLog.addMonitor(new LogMonitorObject(file.getName(), file.getParent(),
tailLogListener, ""+ headerAccessor.getSessionId(), userId)); }}}Copy the codeFor the above several interfaces may not have used his people a little confused, at least WHEN I learn him is so, look at the above code, we have to clarify the logic, to understand why to write this.
Implement the registerStompEndpoints method
First is WebSocketMessageBrokerConfigurer interface, Spring Boot offers a WebSocket configuration interface, only need a simple configuration two times, you can achieve a WebSocket program, there are eight methods in this interface, We only need three of them.
If you don’t give the WebSocket address, how can you continue with the following steps? This is done by implementing the registerStompEndpoints method. You simply add a new “join point” to StompEndpointRegistry via addEndpoint. You can also set an interceptor, which means that when the front end tries to connect, If the back end finds that there is something wrong with the connection, it can refuse to connect to it. This can be done by addInterceptors.
Remember to include withSockJS if you use the SocketJs library.
@Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
registry.addEndpoint("/log")
.addInterceptors(new HttpHandshakeInterceptor())
.setHandshakeHandler(new DefaultHandshakeHandler() {
@Override
protected Principal determineUser(ServerHttpRequest request, WebSocketHandler wsHandler, Map<String, Object> attributes) {
return new StompPrincipal(UUID.randomUUID().toString());
}
})
.withSockJS();
}
Copy the codeSaves the mapping between SessionId and WebSocketSession
This step is for the convenience of management, such as active disconnected, you need to implement configureWebSocketTransport interface, but the SessionId not session ID is generated by the server, but the WebSocket session ID, each connection is different.
The main consideration here is that if the file from the front end does not exist, then the server should be able to disconnect.
@Override
public void configureWebSocketTransport(WebSocketTransportRegistration registration) {
registration.addDecoratorFactory(new WebSocketHandlerDecoratorFactory() {
@Override
public WebSocketHandler decorate(WebSocketHandler webSocketHandler) {
return new WebSocketHandlerDecorator(webSocketHandler) {
@Override
public void afterConnectionEstablished(WebSocketSession session) throws Exception {
log.info("Log monitor WebSocket connection,sessionId={}", session.getId());
mWebSocketManager.add(session);
super.afterConnectionEstablished(session);
}
@Override
public void afterConnectionClosed(WebSocketSession session, CloseStatus closeStatus) throws Exception {
mWebSocketManager.remove(session.getId());
super.afterConnectionClosed(session, closeStatus); }}; }}); }Copy the codeListening to the subscription
The front end then subscribes to a message through Stomp’s API, so how do we receive subscribed events? The SessionSubscribeEvent event is received via the @EventListener annotation.
A front-end subscription requires passing in the log path to monitor. At this point we can get the log path that the WebSocket is listening to.
@EventListener
public void handlerSessionSubscribeEvent(SessionSubscribeEvent sessionSubscribeEvent) {... }Copy the codeStart the tail process
We then start a thread for each WebSocket to execute the tail command.
@Component
public class TailLog {
public static final int MAX_LOG = 3;
private List<LogMonitorExecute> mLogMonitorExecutes = new CopyOnWriteArrayList<>();
/** * Log thread pool */
private ExecutorService mExecutors = Executors.newFixedThreadPool(MAX_LOG);
public void addMonitor(LogMonitorObject object) {
LogMonitorExecute logMonitorExecute = new LogMonitorExecute(object);
mExecutors.execute(logMonitorExecute);
mLogMonitorExecutes.add(logMonitorExecute);
}
public void stopMonitor(String sessionId) {
if (sessionId == null) {
return;
}
for (LogMonitorExecute logMonitorExecute : mLogMonitorExecutes) {
if(sessionId.equals(logMonitorExecute.getLogMonitorObject().getSessionId())) { logMonitorExecute.stop(); mLogMonitorExecutes.remove(logMonitorExecute); }}}public boolean isArriveMaxLog(a) {
returnmLogMonitorExecutes.size() == MAX_LOG; }}Copy the codeThe ultimate executor, where the stop() method is executed when the WebSocket disconnects. Save the mapping between sessionIDS and LogMonitorExecute. When a file changes, it is sent to the corresponding WebSocket.
public class LogMonitorExecute implements Runnable {
private static final Logger log = LoggerFactory.getLogger(LogMonitorExecute.class.getName());
/** * Monitor object */
private LogMonitorObject mLogMonitorObject;
private volatile boolean isStop = false;
/**
* tail 进程对象
*/
private Process mProcess;
public LogMonitorExecute(LogMonitorObject logMonitorObject) {
mLogMonitorObject = logMonitorObject;
}
public LogMonitorObject getLogMonitorObject(a) {
return mLogMonitorObject;
}
@Override
public void run(a) {
try {
String path = Paths.get(mLogMonitorObject.getPath(), mLogMonitorObject.getName()).toString();
log.info("{} start log monitoring on {}", mLogMonitorObject.getSessionId(), path);
mProcess = Runtime.getRuntime().exec(new String[]{"bash"."-c"."tail -f " + path});
InputStream inputStream = mProcess.getInputStream();
BufferedReader mBufferedReader = new BufferedReader(new InputStreamReader(inputStream, "utf-8"));
String buffer = null;
while(! Thread.currentThread().isInterrupted() && ! isStop) { buffer = mBufferedReader.readLine();if(mLogMonitorObject.getTailLogListener() ! =null) {
mLogMonitorObject.getTailLogListener().onNewLine(mLogMonitorObject.getName(), mLogMonitorObject.getPath(), buffer);
continue;
}
break;
}
mBufferedReader.close();
} catch (Exception e) {
e.printStackTrace();
}
log.info("{} exit monitoring {}", mLogMonitorObject.getSessionId(), mLogMonitorObject.getPath() + "/" + mLogMonitorObject.getName());
}
public void stop(a) {
mProcess.destroy();
isStop = true; }}Copy the codeNote that you want to send data to the specified WebSocket, not to the WebSocket subscribed to this path, because using SimpMessagingTemplate when sending data, it can send data to all webSockets subscribed to this path, so that if a browser has 2 monitors open, And if you are monitoring the same log file, each monitor will receive the same two messages.
So use the convertAndSendToUser method instead of convertAndSend, which is why we set the handshake handler to take a name for each WebSocket connection with the setHandshakeHandler.
The front end
<! DOCTYPEhtml>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Log monitoring</title>
<style>
body {
background: # 000000;
color: #ffffff;
}
.log-list {
color: #ffffff;
font-size: 13px;
padding: 25px;
}
</style>
</head>
<body>
<div class="container">
<div class="log-list">
</div>
</div>
<script src="https://cdn.jsdelivr.net/npm/sockjs-client@1/dist/sockjs.min.js"></script>
<script src="/lib/stomp/stomp.min.js"></script>
<script src="https://lib.sinaapp.com/js/jquery/2.0.2/jquery-2.0.2.min.js"></script>
<script>
var socket = new SockJS('/socket-log? a=a');
stompClient = Stomp.over(socket);
stompClient.connect({}, function (frame) {
stompClient.subscribe('/user/topic/path'+getQueryVariable("path"), function (greeting) {
console.log("a" + greeting)
let item = $("<div class='log-line'></div>");
item.text(greeting.body)
$(".log-list").append(item);
$("html, body").animate({scrollTop: $(document).height()}, 0);
});
});
function getQueryVariable(variable) {
var query = window.location.search.substring(1);
var vars = query.split("&");
for (var i = 0; i < vars.length; i++) {
var pair = vars[i].split("=");
if (pair[0] == variable) {
return encodeURIComponent(pair[1]); }}return (false);
}
</script>
</body>
</html>
Copy the codeThe effect
Here are the logs for starting and shutting down Tomcat.
How to send data without using SimpMessagingTemplate
If we don’t use SimpMessagingTemplate, first we need to get the corresponding WebSocketSession, which has a sendMessage method to send data, but of type WebSocketMessage, Spring Boot has several default implementations, such as TextMessage for sending text messages.
However, if Stomp is used, it is not enough to simply use it to send. Although the data can be sent, the format is not correct and Stomp cannot parse it. Therefore, we need to send it according to Stomp format.
However, after searching, I could not find relevant information, so I looked at his source code, in which I designed the class StompEncoder. From the name, I know that it is a tool for Stomp coding. Stomp consists of three parts: commands, headers, and message bodies. Commands are as follows:
CONNECT
SEND
SUBSCRIBE
UNSUBSCRIBE
BEGIN
COMMIT
ABORT
ACK
NACK
DISCONNECT
Copy the codeFollowing the command line is the header, which is in the form of a key-value pair, and finally the message body, which ends with a null character.
Below is the necessary format, send StompEncoder also won’t be able to code, will throw an exception, as to the why do you write, detailed’ll have to see StompEncoderde. WriteHeaders method, there are several validation, this kind of written entirely by his force.
StompEncoder stompEncoder = new StompEncoder();
byte[] encode = stompEncoder.encode(createStompMessageHeader(),msg.getBytes());
webSocketSession.sendMessage(new TextMessage(encode));
private HashMap<String, Object> createStompMessageHeader(a) {
HashMap<String, Object> hashMap = new HashMap<>();
hashMap.put("subscription", createList("sub-0"));
hashMap.put("content-type", createList("text/plain"));
HashMap<String, Object> stringObjectHashMap = new HashMap<>();
stringObjectHashMap.put("simpMessageType", SimpMessageType.MESSAGE);
stringObjectHashMap.put("stompCommand", StompCommand.MESSAGE);
stringObjectHashMap.put("subscription"."sub-0");
stringObjectHashMap.put("nativeHeaders", hashMap);
return stringObjectHashMap;
}
private List<String> createList(String value) {
List<String> list = new ArrayList<>();
list.add(value);
return list;
}
Copy the codeWhy does tail -f fail
Tail -f: echo test>>xx. TXT: echo test>>xx.
So what’s so weird about this place?
In fact, tail-f tracks files as they move and change names, because it tracks file descriptors. To quote wikipedia:
The file descriptor is formally a non-negative integer. In fact, it is an index value that points to the record table of open files that the kernel maintains for each process. When a program opens an existing file or creates a new file, the kernel returns a file descriptor to the process. In programming, some low-level programming tends to revolve around file descriptors. However, the concept of file descriptors is usually only applicable to operating systems such as UNIX and Linux.
Tail -f is executed to generate a process that can view the open file descriptor in /proc/pid/fd. Here’s a GIF.
TXT is created in terminal 1, followed by tail -f tracing, followed by appending a line of data to terminal 2. You can see that terminal 1 is printable.
Then watch the magic scene, rename mv in terminal 2, then add a new line to the renamed file, you will find that terminal 1 will still print.
If you look at the file descriptor in this process, it is not surprising that, in the command below, shows the descriptor is track 3 / home/HouXinLin/test/tail / 2. TXT.
hxl@hxl-PC:/home/HouXinLin/test/tail$ ps -ef |grep 1.txt
hxl 1368 29021 0 09:02 pts/0 00:00:00 grep 1.txt
hxl 20298 29672 0 09:00 pts/6 00:00:00 tail -f 1.txt
hxl@hxl-PC:/home/HouXinLin/test/tail$ ls -l /proc/20298/ fd total consumption0
lrwx------ 1 hxl hxl 64 3month16 09:02 0 -> /dev/pts/6
lrwx------ 1 hxl hxl 64 3month16 09:02 1 -> /dev/pts/6
lrwx------ 1 hxl hxl 64 3month16 09:02 2 -> /dev/pts/6
lr-x------ 1 hxl hxl 64 3month16 09:02 3 -> /home/HouXinLin/test/tail/2.txt
lr-x------ 1 hxl hxl 64 3month16 09:02 4 -> anon_inode:inotify
hxl@hxl-PC:/home/HouXinLin/test/tail$
Copy the codeHowever, if we edit the file through vim, etc., the file descriptor will be recorded as deleted, even if the file does exist, then the appending to the 2.txt file will be invalid.
hxl@hxl-PC:/home/HouXinLin/test/tail$ vim 2.txt
hxl@hxl-PC:/home/HouXinLin/test/tail$ ls -l /proc/20298/ fd total consumption0
lrwx------ 1 hxl hxl 64 3month16 09:02 0 -> /dev/pts/6
lrwx------ 1 hxl hxl 64 3month16 09:02 1 -> /dev/pts/6
lrwx------ 1 hxl hxl 64 3month16 09:02 2 -> /dev/pts/6
lr-x------ 1 hxl hxl 64 3month16 09:02 3 -> /home/HouXinLin/test/tail/2.txt~ (deleted)
lr-x------ 1 hxl hxl 64 3month16 09:02 4 -> anon_inode:inotify
hxl@hxl-PC:/home/HouXinLin/test/tail$
Copy the codeFinally, try tail-f?