As an experienced and responsible person, the first thing I do at work every day is to connect to the production server, warm up with the command of top free PS like a tiger, and then summarize the status of business services, system indicators, etc. Only when I see everything is normal can I feel at ease.

I do not know how many conscientious friends, every day to do such a mechanical and simple thing. To be honest, it’s no different than a mechanical dog. In the spirit of doing things seriously, the pursuit of the ultimate attitude, then why can’t we create a mechanical dog like this?

Those of you who have been concerned about ape wai should know that in previous articles, ape Wai has mentioned the monitoring application of the guardian service – watchdog. Good, want to build the mechanical dog, is the watchdog.

Next, I will give you a brief talk about the background source of the watchdog and the realization of ideas, hoping to give friends who are looking for solutions to guard service monitoring, a point of realization of ideas. The following content is very brain-burning, please sit firmly and hold well.

1. What is a watchdog?

In a composed of single chip microcomputer system, due to the single-chip computer work, often will be affected by interference from external electromagnetic field, cause program run to fly, and into an infinite loop, the normal operation of the program is interrupted, controlled by single chip microcomputer system cannot continue to work, can cause the system to a standstill, unpredictable consequences, Therefore, for the consideration of real-time monitoring of the running state of the single chip, a special chip for monitoring the running state of the single chip program was produced, called “watchdog” — extracted from “Baidu Encyclopedia”.

I believe there are many friends, see above a paragraph of inhuman description, would like to directly delete this article xiong Wen, run away. Everyone, don’t panic, ape outside again give a chestnut close to life to explain.

Apes domesticated a dog, often play with the child, the child from time to time to throw a handful of dog food to dog; If the child falls asleep and has not spread dog food for a long time, the dog takes the initiative to wake the child up. Continue to let the child spread dog food, then dog eat the child spread dog food, happy to bark; If the dog runs away and the child stops spreading dog food, the child can go to sleep.

Peel the chestnuts, the expert sees the way, the layman sees the fun.

An ape keeps a dog in his house. He often plays with his child, and the child throws a handful of dog food to the dog from time to time. There are two processes: the child process (application service) and the parent process (daemon). Heartbeat communication between the child process and the parent process is maintained.

Quote: If a child falls asleep and has not spread dog food for a long time, dog will take the initiative to wake the child up. The heartbeat communication between the child process and its parent process is interrupted. The parent process is responsible for starting the child process and completing the service daemon of the child process.

Quote: Continue to let the child spread dog food, then dog eat the child spread dog food, happy to bark; The child process communicates with the parent process in real time, and the parent process monitors the status of the child process in real time, and gives alarm notification in real time.

Quote: If the dog runs away and the child stops spreading dog food, the child can go to sleep. When the parent process becomes down, communication between the child process and the parent process cannot be maintained, and the child process also exits.

Do not know friends through the above chestnut analysis, how much to understand the watchdog? In order to make it clear to you, ape again to give you a clear picture.

As shown in the figure above, zouwai mainly positioned the watchdog as a monitoring system integrating service guarding, index collection, log collection and automatic alarm. To put it bluntly, with a watchdog, my mom doesn’t have to worry about my app service anymore.

So much has been laid out, but how can it be achieved?

2. How to implement it?

Ape outside first to draw a brain map for everyone, mainly divided into five steps, I hope friends follow ape outside footsteps, mo lag behind.

I believe that the vast majority of friends through the ape outside the brain map, has to understand the watchdog application. Ape a little more elaboration:

Implementation: The watchdog app took some detours in the implementation process. At the beginning of the selection of technology, considering that the watchdog should have zero intrusion and low coupling for the application, javaAgent implantation was adopted to implement it. However, what we never expected was that the development and debugging process was quite troublesome. Once problems were encountered, partners would check for a long time. The second iteration was carried out in a plug-in way.

Service daemon: Use the ProcessBuilder class newly added in J2SE 1.5 java.lang package to start, stop, and restart the child process (complete the daemon function of the child service).

Counter collection: Connects to sub-services in JMX mode to obtain the memory usage, CPU usage, and number of threads of sub-services.

Log collection: Mainly on flume’s shoulder, integrated into the project and secondary development, storage using ElasticSearch.

Alarm notification: mainly provide email notification, QQ notification, micro – reliance notification.

Each of the implementation details of the ape is no longer carried out in depth, interested friends welcome to pay attention to the wechat public number four ape outside (Si-Yuanwai), background messages for communication.

Rory has written so much. Rome was not built in a day, but all the pits and hardships it had trodden on its way to realization came back to me in an instant. However, the watchdog app has been promoted and used in production, and has been widely praised since it was launched. Now, the first thing we do every day is to make coffee instead of the original system inspection, and the happiness index has risen sharply.

Finally, ape outside wants to say: do simple things repeatedly, you are an expert; Repeat with heart, you are a winner. Yes, there’s nothing wrong with that, but while you’re an expert, you might as well innovate a little and see if there’s a technology wheel that can be used to change the way machines work, improve productivity, and increase happiness.

Welcome to my public account: Outside the Four Apes. I have prepared some hand-written high quality PDFS, some sponsored by friends and some of my own, which you can get for free: Simple Java multithreading, HTTP super complete summary, Java basic core summary, programmers must know the core knowledge, resume interview and salary super complete dry things. Don’t look at the number is not much, but the articles are dry goods, read all said very liver.

How to get it: Scan the code and reply to 666 in the background of the official account