Use ancillary services to build your own smart video surveillance system

In recent months, two new members have been added to the family, NIDA and Water. NIDA is a Chinese farm cat who was adopted by a female cat the night before Typhoon Nitan landed in July. NIDA has a fierce personality and likes to attack and bite human legs as prey. Active, like to use the mouth to “bite” the cat, NIDA toys for their own, at first also often steal to eat cat food, water…. NIDA is helpless and usually gets drool all over her. What about her dignity as a cat star? But NIDA runs up and down, and Water is left in the dust. As you can imagine, every day when he comes home from work, he is greeted by his almost ransacked home, Water’s 💩, and Water’s pee. Put the two things at home or at work every day is a little bit worried, buy a video monitor, said also need a less controlled (can help them buy a lot of snacks), combined with own hand we have a spare broken screen mobile phone (also to change the screen more than 100), so I would want to own a remote video monitoring system, Monitor your home if you need to

idea

In order to realize video communication, it is enough to use the video call function provided by mobile QQ, which is stable enough, so the spare QQ which has been wasted for many years can finally be used (don’t envy me this man who has two QQ). The communication object is divided into Client and Server. As for the communication model, the Client sends a specific command to the Server, and the Server parses the command of the Client. For example, the Client initiates A QQ video chat. The Client only needs to wait and receive the video chat, and finally can monitor the image of the camera on the Server. It still looks SO EASY, SO let’s do it

implementation

The main problem is how to achieve automatic operation without manual intervention, and the auxiliary service function of the system can solve this problem well. I believe that most developers know that we can use the auxiliary service to write wechat red envelope snatchplugin. For details, see this project. AccessibilityService allows you to listen for changes in one or more types of events (notification center, window content, window state, focus change, etc.)

State transition

AccessibilityService Automates operations using AccessibilityService (AccessibilityService)

It can be seen that there are many scenes, and each scene requires us to complete specific operations. For example, we need to monitor the arrival of QQ messages on the lock screen, and we need to check whether they are from our Client (IN the project, I used “WaterMonitor: QQ number 】 is the mark, through the Server to modify the Client’S QQ remarks processing, such a format is also convenient to obtain the need for video call QQ contacts), and the request command, if these are in line with the simulation HOME button to enter the lock screen interface, in the lock screen interface also need to simulate the operation to enter the unlock interface, And unlock interface input the correct password to unlock, for this kind of conversion in different scenarios (state) and make the corresponding processing scenario, I don’t want to pass the If/else to determine the current status, and treatment, this greatly increases the coupling of the program, and after considering the likely when open the QQ login prompt expired, Then I need to add an automatic login detection and operation. For decoupling, the state machine mode works just fine here. Here is the state diagram of the program:

The following is a brief description of the responsibilities of each state

state	responsibility
IdleState	Check Client commands and unlock the screen to open the QQ chat interface
QQChatState	Check whether the chat interface, find ➕ key, click to bring up more functions panel
StartVideoState	The video call button is detected and clicked to initiate a video call
EndingSate	End of call, check whether the last Item in the list is end of call, Cancel, reject, and reset the status to IdleState

I’m going to pick IdleState for a simple analysis

Configuration of ancillary services

public class VideoAccessibilityService extends AccessibilityService implements IMonitorService { private MonitorState mCurState; @Override protected void onServiceConnected() { super.onServiceConnected(); registerScreenReceiver(); AccessibilityServiceInfo info = new AccessibilityServiceInfo(); info.eventTypes = TYPE_WINDOW_CONTENT_CHANGED | TYPE_WINDOWS_CHANGED | TYPE_WINDOW_STATE_CHANGED | TYPE_NOTIFICATION_STATE_CHANGED; info.packageNames = new String[]{Constant.QQ_PKG}; / /... this.setServiceInfo(info); setState(new IdleState(this)); } @Override public void onAccessibilityEvent(AccessibilityEvent accessibilityEvent) { if (mCurState ! = null) { mCurState.handle(accessibilityEvent); }} / /... }Copy the code

MCurState records the current state and sends events to onAccessibilityEvent whenever the onAccessibilityEvent method calls back. OnAccessibilityEvent listens for events configured in onServiceConnected methods. To monitor the event type TYPE_WINDOW_CONTENT_CHANGED | TYPE_WINDOWS_CHANGED | TYPE_WINDOW_STATE_CHANGED | TYPE_NOTIFICATION_STATE_CHANGED, Corresponding to the content of the window (such as adding a View), the display of the window (when it is displayed in the foreground), the state of the window (Dialog popup causes the window to lose focus, etc.) and the event type of the state change of the notification bar. The monitored package name is the package name of QQ, and the change of the notification bar is not affected by the package name

The processing of IdleState

/** * initial state, * Change to monitor QQ new message (LockScreen, Notification, QQ App) * Created by chensuilun on 16-10-9. */ public class IdleState extends MonitorState { //... @Override public void handle(AccessibilityEvent accessibilityEvent) { AccessibilityNodeInfo nodeInfo = mContextService.getWindowNode(); if (nodeInfo == null) { return; } if (isLockScreenMonitorMsg(nodeInfo, accessibilityEvent) || isNotificationMonitorMsg(nodeInfo, accessibilityEvent)) { if (AppUtils.isInLockScreen()) { // back press RootCmd.execRootCmd("input keyevent " + KeyEvent.KEYCODE_BACK); // press HOME rootcmd. execRootCmd("sleep 0.1 && input keyevent "+ keyevent.keycode_HOME); unlockScreen(nodeInfo); } final String qqNumber = retrieveQQNumber(nodeInfo, accessibilityEvent); mContextService.setState(new QQChatState(mContextService)); AppApplication.postDelay(new Runnable() { @Override public void run() { AppUtils.openQQChat(qqNumber); }}, 1000); } } /** * retract monitor cmd from notification * * @param nodeInfo * @param accessibilityEvent * @return */ private boolean isNotificationMonitorMsg(AccessibilityNodeInfo nodeInfo, AccessibilityEvent accessibilityEvent) { if (accessibilityEvent.getEventType() == TYPE_NOTIFICATION_STATE_CHANGED) { Parcelable data = accessibilityEvent.getParcelableData(); if (data instanceof Notification) { if (((Notification) data).tickerText ! = null) { return (((Notification) data).tickerText.toString().startsWith(MONITOR_TAG) && ((Notification) data).tickerText.toString().endsWith(Constant.MONITOR_CMD_VIDEO)); } } } return false; } /** * @param nodeInfo * @param accessibilityEvent * @return If from notification ,msg format :{@link Constant#MONITOR_TAG} + ":real QQ No: "+{@link Constant#MONITOR_CMD_VIDEO} */ private String retrieveQQNumber(AccessibilityNodeInfo nodeInfo, AccessibilityEvent accessibilityEvent) { if (accessibilityEvent.getEventType() == TYPE_NOTIFICATION_STATE_CHANGED) { Parcelable data = accessibilityEvent.getParcelableData(); if (data instanceof Notification) { if (((Notification) data).tickerText ! = null) { return ((Notification) data).tickerText.toString().split(":")[1]; } } } else { List nodeInfos = nodeInfo.findAccessibilityNodeInfosByText(MONITOR_TAG); if (! AppUtils.isListEmpty(nodeInfos)) { String tag; for (AccessibilityNodeInfo info : nodeInfos) { tag = (String) info.getText(); if (! TextUtils.isEmpty(tag) && tag.contains(MONITOR_TAG)) { return tag.substring(Constant.MONITOR_TAG.length()); } } } } return Privacy.QQ_NUMBER; } /** * receive monitor cmd in LockScreen * * @param nodeInfo * @param accessibilityEvent * @return */ private boolean isLockScreenMonitorMsg(AccessibilityNodeInfo nodeInfo, AccessibilityEvent accessibilityEvent) { if (AppUtils.isInLockScreen() && Constant.QQ_PKG.equals(nodeInfo.getPackageName()) && TYPE_WINDOW_CONTENT_CHANGED == accessibilityEvent.getEventType()) {  if (! AppUtils.isListEmpty(nodeInfo.findAccessibilityNodeInfosByText(MONITOR_TAG)) && ! AppUtils.isListEmpty(nodeInfo.findAccessibilityNodeInfosByText(Constant.MONITOR_CMD_VIDEO))) { return true; } } return false; } /** * unlock @param nodeInfo */ private void unlockScreen(AccessibilityNodeInfo nodeInfo) {unlockutils.unlock (); }}Copy the code

IdelState handles changes in the window of the QQ package name or changes in the notification bar. IsLockScreenMonitorMsg receives changes in the window of the QQ package name on the lock screen. It determines whether it has received the command from the Client by looking for the WaterMonitor flag and command 1. IsNotificationMonitorMsg detects the changes in the notification bar. If command 1 is received from the Client marked WaterMonitor:111, isNotificationMonitorMsg detects the changes in the notification bar. WaterMonitor:111 1 is obtained by reading the content of the notification bar. If it is a video command on the Client side, then determine whether the screen is locked and unlock it. Otherwise, directly find the QQ of the contact, open the QQ chat interface and change the state to QQChatState. QQChatState will handle the following matters. It does not monitor the message list and new messages from the chat panel of QQ’s main program, mainly because it is difficult to determine whether the new command has been processed, but it does not affect the use of the program. This is because the state is initialized and the screen is turned off either when the screen goes off or when the chat ends

The same goes for other states

Root and screen unlock

Found in the process of development, the use of pure service service is not enough, is unable to unlock the screen, the lock screen is mostly a custom View, and generally do not support the auxiliary function, this is the biggest problem encountered in the development, even thought about if do not lock screen give up well, disable safety lock screen can be used to easily to avoid this problem, But for me, I can’t accept such restrictions. Finally, in order to unlock, I found that the user can simulate key pressing, touch and other operations through ADB input command. For details, please see here

Sleep 0.1 && input keyevent 3 input swipe 655 1774 655 874 sleep 1 && input tap 612 726 sleep 0.1 && input tap 813 1000 Sleep 0.1 && input tap 813 1000 sleep 0.1 && input tap 255 1000 quitCopy the code

Swipe is equal to 3, which means that this is a HOME button event, so the first line is equivalent to hitting the HOME button. For more KEYCODE, see Android.view. keyevent. Swipe is the swipe operation that simulates a finger swipe from 655,1774 to 655. 874), that is, the finger swipe, mainly to enter the unlock interface, tap is the click action, followed by the click coordinates, so the next four tap, is the simulation of the unlock interface of some numbers, quit is the program itself used to determine the end of the script, not adb command. To adapt to different mobile phones, separate the unlock script and store it in the root directory of the SD card. The file name is MonitorUnlock. TXT

How to determine the coordinates, in fact, very simple, open the developer mode – pointer position to see their actual operation of the coordinate value

Also, in order to be able to execute ADB commands, Root is required

The last

In order to ensure that the program and QQ can run in the background, so remember to add to the system to clean up the white list oh, and if the use of domestic ROM, it is best to add the program to the system to open the boot, you do not need to restart each manual open auxiliary services project has been uploaded to Github, welcome Start💕

The effect

Attached are two handsome master photos