An in-depth understanding of the JNI family of input systems
Based on the Android 8.1
preface
In the Android input system (3) InputReader processing type and InputDispatcher distribution process, due to the length of the article, there is still a part of the InputDispatcher distribution process is not explained, this part is the process of event distribution to the target window.
1. Find the right distribution target for the event
Let’s review on an article explaining InputDispatcher dispatchOnceInnerLocked function: frameworks/native/services/inputflinger InputDispatcher. CPP
void InputDispatcher::dispatchOnceInnerLocked(nsecs_t* nextWakeupTime) {
...
DropReason dropReason = DROP_REASON_NOT_DROPPED;/ / 1.switch (mPendingEvent->type) {/ / 2.case EventEntry::TYPE_MOTION: {
MotionEntry* typedEntry = static_cast<MotionEntry*>(mPendingEvent);
// If there is no prompt response to the window switch operation
if (dropReason == DROP_REASON_NOT_DROPPED && isAppSwitchDue) {
dropReason = DROP_REASON_APP_SWITCH;
}
// The event expired
if (dropReason == DROP_REASON_NOT_DROPPED
&& isStaleEventLocked(currentTime, typedEntry)) {
dropReason = DROP_REASON_STALE;
}
// Block other Windows from getting events
if (dropReason == DROP_REASON_NOT_DROPPED && mNextUnblockedEvent) {
dropReason = DROP_REASON_BLOCKED;
}
done = dispatchMotionLocked(currentTime, typedEntry,
&dropReason, nextWakeupTime);/ / 3
break;
}
default:
ALOG_ASSERT(false);
break; }... }Copy the codeThere are five main things that dispatchOnceInnerLocked does, and only one of them is intercepted here: event discarding. The dropReason at comment 1 represents the reason why the event was dropped. Its default value is DROP_REASON_NOT_DROPPED, indicating that the event was not dropped. Note 2 distinguishes mPendingEvent according to the type of mPendingEvent, which mainly intercepts the processing of Motion type. After conditional statement filtering, the dispatchMotionLocked function at comment 3 is called to find the appropriate window for the Motion event. frameworks/native/services/inputflinger/InputDispatcher.cpp
bool InputDispatcher::dispatchMotionLocked(
nsecs_t currentTime, MotionEntry* entry, DropReason* dropReason, nsecs_t* nextWakeupTime) {
if (! entry->dispatchInProgress) {
// Indicates that the distribution process is currently underway
entry->dispatchInProgress = true;
logOutboundMotionDetailsLocked("dispatchMotion - ", entry);
}
// If the event needs to be discarded, return true and no window will be found for the event
if(*dropReason ! = DROP_REASON_NOT_DROPPED) {/ / 1
setInjectionResultLocked(entry, *dropReason == DROP_REASON_POLICY
? INPUT_EVENT_INJECTION_SUCCEEDED : INPUT_EVENT_INJECTION_FAILED);
return true;
}
bool isPointerEvent = entry->source & AINPUT_SOURCE_CLASS_POINTER;
// The target window information list is stored in the inputTargets
Vector<InputTarget> inputTargets;/ / 2
bool conflictingPointerActions = false;
int32_t injectionResult;
if (isPointerEvent) {
// Handle click-like events, such as touching the screen
injectionResult = findTouchedWindowTargetsLocked(currentTime,
entry, inputTargets, nextWakeupTime, &conflictingPointerActions);/ / 3
} else {
// Handle non-touch events, such as trackballs
injectionResult = findFocusedWindowTargetsLocked(currentTime,
entry, inputTargets, nextWakeupTime);/ / 4
}
// The input event is suspended, indicating that the window is found and the window is not responding
if (injectionResult == INPUT_EVENT_INJECTION_PENDING) {
return false;
}
setInjectionResultLocked(entry, injectionResult);
// The input event was not successfully distributed, indicating that the appropriate window was not found
if(injectionResult ! = INPUT_EVENT_INJECTION_SUCCEEDED) {if(injectionResult ! = INPUT_EVENT_INJECTION_PERMISSION_DENIED) { CancelationOptions::Mode mode(isPointerEvent ? CancelationOptions::CANCEL_POINTER_EVENTS : CancelationOptions::CANCEL_NON_POINTER_EVENTS);
CancelationOptions options(mode, "input event injection failed");
synthesizeCancelationEventsForMonitorsLocked(options);
}
return true;
}
// The distribution target is added to the inputTargets list
addMonitoringTargetsLocked(inputTargets);/ / 5
// Dispatch the motion.
if (conflictingPointerActions) {
CancelationOptions options(CancelationOptions::CANCEL_POINTER_EVENTS,
"conflicting pointer actions");
synthesizeCancelationEventsForAllConnectionsLocked(options);
}
// Distributes events to targets in the inputTargets list
dispatchEventLocked(currentTime, entry, inputTargets);/ / 6
return true;
}
Copy the codeComment 1 indicates that the event should be discarded. In this case, true is returned and no window is found for the event. The dispatch task is not completed and the dispatch will be attempted again in the next InputDispatcherThread loop. In Comments 3 and 4, the events in the form of click and non-touch are processed, and the result of event processing is submitted to injectionResult. If the value of injectionResult is INPUT_EVENT_INJECTION_PENDING, the window is found and the input event is pending without response, false is returned. If the value of injectionResult is not INPUT_EVENT_INJECTION_SUCCEEDED, the appropriate window was not found and the input event was not successfully distributed, which returns true. Comment 5 adds the distributed target to the inputTargets list, and finally the event is distributed to the target in the inputTargets list at comment 6. Storage can be seen from the 2 inputTargets list is InputTarget structure: frameworks/native/services/inputflinger InputDispatcher. H
struct InputTarget {
enum {
// This flag indicates that the event is being delivered to the foreground application
FLAG_FOREGROUND = 1 << 0.// This flag indicates that the MotionEvent is in the target region
FLAG_WINDOW_IS_OBSCURED = 1 << 1. };//inputDispatcher communicates with the target window
sp<InputChannel> inputChannel;/ / 1
// Event dispatches flags
int32_t flags;
// The offset of the screen coordinate system with respect to the target window coordinate system
float xOffset, yOffset;/ / 2
// The scaling factor of the screen coordinate system relative to the target window coordinate system
float scaleFactor;/ / 3
BitSet32 pointerIds;
}
Copy the codeThe InputTarget structure can be said to be the converter of inputDispatcher and target window. It is divided into two parts, one is the token of inputDispatcher interaction with target window stored in enumeration, and the other part is the parameter of inputDispatcher interaction with target window. For example, the inputChannel in comment 1 is actually a SocketPair. SocketPair is used for two-way communication between processes. This is perfect for communication between inputDispatcher and the target window because inputDispatcher not only dispatches events to the target window, InputDispatcher also needs to get the target window’s response to the event. XOffset and yOffset at comment 2, the offset of the screen coordinate system with respect to the target window coordinate system, and the coordinates stored in MotionEntry(MotionEvent) are the screen coordinate system, so the parameters at Comment 2 and Comment 3 are needed to convert the screen coordinate system to the target window coordinate system.
2. Handle click events
In comments 3 and 4 of InputDispatchMotionLocked, click and non-touch Motion events are processed, respectively. Since non-touch events are not very common, click events are parsed here. InputDispatcher findTouchedWindowTargetsLocked function such as more than 400 lines, intercepting the need to understand here, and is divided into two parts to explain. frameworks/native/services/inputflinger/InputDispatcher.cpp
1. Part1 findTouchedWindowTargetsLocked function:
int32_t InputDispatcher::findTouchedWindowTargetsLocked(nsecs_t currentTime,
const MotionEntry* entry, Vector<InputTarget>& inputTargets, nsecs_t* nextWakeupTime,
bool* outConflictingPointerActions) {
...
if (newGesture || (isSplit && maskedAction == AMOTION_EVENT_ACTION_POINTER_DOWN)) {
// Get the coordinates from MotionEntry
int32_t pointerIndex = getMotionEventActionPointerIndex(action);
int32_t x = int32_t(entry->pointerCoords[pointerIndex].
getAxisValue(AMOTION_EVENT_AXIS_X));
int32_t y = int32_t(entry->pointerCoords[pointerIndex].
getAxisValue(AMOTION_EVENT_AXIS_Y));
sp<InputWindowHandle> newTouchedWindowHandle;
bool isTouchModal = false;
size_t numWindows = mWindowHandles.size();/ / 1
// Walk through the window to find the touched window and the external target outside the window
for (size_t i = 0; i < numWindows; i++) {/ / 2
// Get the windowHandle representing Windows in InputDispatcher
sp<InputWindowHandle> windowHandle = mWindowHandles.itemAt(i);
// Get windowInfo
const InputWindowInfo* windowInfo = windowHandle->getInfo();
if(windowInfo->displayId ! = displayId) {// If displayId does not match, start the next loop
continue;
}
// Get the window flag
int32_t flags = windowInfo->layoutParamsFlags;
// if the window is visible
if (windowInfo->visible) {
// If the window flag is not FLAG_NOT_TOUCHABLE
if (! (flags & InputWindowInfo::FLAG_NOT_TOUCHABLE)) {
// If the window is focusable or flag is not FLAG_NOT_FOCUSABLE, the window is "touchable".
isTouchModal = (flags & (InputWindowInfo::FLAG_NOT_FOCUSABLE
| InputWindowInfo::FLAG_NOT_TOUCH_MODAL)) == 0;/ / 3
// If the window is "touchable mode" or the coordinate point falls on the window
if (isTouchModal || windowInfo->touchableRegionContainsPoint(x, y)) {
newTouchedWindowHandle = windowHandle;/ / 4
break; // found touched window, exit window loop}}if (maskedAction == AMOTION_EVENT_ACTION_DOWN
&& (flags & InputWindowInfo::FLAG_WATCH_OUTSIDE_TOUCH)) {
// Place the eligible window in TempTouchState for subsequent processing.
mTempTouchState.addOrUpdateWindow(
windowHandle, InputTarget::FLAG_DISPATCH_AS_OUTSIDE, BitSet32(0));/ / 5}}}Copy the codeStart by getting the coordinates from MotionEntry for later filtering Windows. Note 1 to get the number of inputwindowhandles in the list mWindowHandles, where InputWindowInfo is stored. Again in InputWindowInfo contains a WindowManager. LayoutParams defined Window signs, sign on the Window to see Android parsing WindowManager (2) the Window properties of this article. In addition to window flags, InputWindowInfo also contains InputChannel and various window properties. InputWindowInfo describes the properties of a window that can receive input events. In this way, InputWindowHandle is similar to Windows State in WMS. Informally, WindowState is used to represent Windows in WMS, and InputWindowHandle is used to represent Windows in the input system. So how does the input system get window information? This is because mWindowHandles lists are WMS updates to InputDispatcher. Note 2 begins to iterate over the Windows in the mWindowHandles list, finding the touched window and the external target outside the window. At comment 3, if the window is focusable or flag is not FLAG_NOT_FOCUSABLE, the window is touchable. After several layers of filtering, if the window is in “touchable mode” or the coordinates are on the window, the windowHandle is assigned to “newTouchedWindowHandle” at comment 4. Finally, at comment 5, add newTouchedWindowHandle to TempTouchState for subsequent processing.
2. Part2 findTouchedWindowTargetsLocked function:
.// Make sure that all foreground Windows touched are ready for new input
for (size_t i = 0; i < mTempTouchState.windows.size(); i++) {
const TouchedWindow& touchedWindow = mTempTouchState.windows[i];
if (touchedWindow.targetFlags & InputTarget::FLAG_FOREGROUND) {
// Check if the window is ready to receive more input
String8 reason = checkWindowReadyForMoreInputLocked(currentTime,
touchedWindow.windowHandle, entry, "touched");/ / 1
if(! reason.isEmpty()) {/ / 2
// If the window is not ready, the reason is assigned to injectionResult
injectionResult = handleTargetsNotReadyLocked(currentTime, entry,
NULL, touchedWindow.windowHandle, nextWakeupTime, reason.string());/ / 3
// Skip to the Unresponsive TAB
goto Unresponsive;/ / 3}}}...// The window has been searched successfully
injectionResult = INPUT_EVENT_INJECTION_SUCCEEDED;/ / 5
// Iterate over the window in TempTouchState
for (size_t i = 0; i < mTempTouchState.windows.size(); i++) {
const TouchedWindow& touchedWindow = mTempTouchState.windows.itemAt(i);
// Generate InputTargets for each window in mTempTouchState
addWindowTargetLocked(touchedWindow.windowHandle, touchedWindow.targetFlags,
touchedWindow.pointerIds, inputTargets);/ / 6
}
// In the next iteration, remove the external window or hover touch windowmTempTouchState.filterNonAsIsTouchWindows(); . Unresponsive:/ / reset TempTouchState
mTempTouchState.reset();
nsecs_t timeSpentWaitingForApplication = getTimeSpentWaitingForApplicationLocked(currentTime);
updateDispatchStatisticsLocked(currentTime, entry,
injectionResult, timeSpentWaitingForApplication);
#if DEBUG_FOCUS
ALOGD("findTouchedWindow finished: injectionResult=%d, injectionPermission=%d, "
"TimeSpentWaitingForApplication = % 0.1 FMS",
injectionResult, injectionPermission, timeSpentWaitingForApplication / 1000000.0);
#endif
return injectionResult;
}
Copy the codeComment 1 checks that the window is ready to receive more input and assigns the result to Reason. 2, if the value is not empty, reason that the window can’t receive more input, comment 3 handleTargetsNotReadyLocked function will be unable to receive more input, assigned to injectionResult, its internal processing time will also have a window function, If a timeout occurs (default is 5 seconds), ANR is reported and nextWakeupTime is set to LONG_LONG_MIN, forcing InputDispatcherThread to be woken up immediately on the next loop and InputDispatcher to restart the distribution of input events. At this point, the value of injectionResult is INPUT_EVENT_INJECTION_PENDING. Because the window cannot receive any more input, it will call the goTO statement at comment 4 to jump to the Unresponsive tag, which will call the reset function of TempTouchState to reset the TempTouchState. If the code has reached comment 5, the window has been found successfully, and the Windows in TempTouchState are iterated, generating inputTargets at comment 6 for each window in TempTouchState. In the first section, comment 6 of InputDispatcher’s dispatchMotionLocked function calls InputDispatcher’s dispatchEventLocked function to dispatch events to distribution targets in the inputTargets list. Let’s take a look at how this works.
3. Send events to the target window
The dispatchEventLocked functions of InputDispatcher are shown below. frameworks/native/services/inputflinger/InputDispatcher.cpp
void InputDispatcher::dispatchEventLocked(nsecs_t currentTime,
EventEntry* eventEntry, const Vector<InputTarget>& inputTargets) {
#if DEBUG_DISPATCH_CYCLE
ALOGD("dispatchEventToCurrentInputTargets");
#endif
ALOG_ASSERT(eventEntry->dispatchInProgress); // should already have been set to true
pokeUserActivityLocked(eventEntry);
// Iterate over the list of inputTargets
for (size_t i = 0; i < inputTargets.size(); i++) {
const InputTarget& inputTarget = inputTargets.itemAt(i);
// Get the index of Connection based on inputChannel inside inputTarget
ssize_t connectionIndex = getConnectionIndexLocked(inputTarget.inputChannel);/ / 1
if (connectionIndex >= 0) {
// Get the Connection stored in the mConnectionsByFd container
sp<Connection> connection = mConnectionsByFd.valueAt(connectionIndex);
// Start the event sending loop according to inputTarget
prepareDispatchCycleLocked(currentTime, connection, eventEntry, &inputTarget);/ / 2
} else{#if DEBUG_FOCUS
ALOGD("Dropping event delivery to target with channel '%s' because it "
"is no longer registered with the input dispatcher.",
inputTarget.inputChannel->getName().string());
#endif}}}Copy the codeWalk through the list of inputTargets and get each inputTarget. At comment 1, get the index of Connection according to inputChannel inside inputTarget. This index is then used as the Key to retrieve the Connection in the mConnectionsByFd container. Connection can be understood as the Connection between InputDispatcher and the target window, which contains the state of the Connection, InputChannel, InputWindowHandle, event queue, etc. 2 place call prepareDispatchCycleLocked function according to the current inputTarget, began to send cycle events. The inputChannel in inputTarget is used for interprocess communication with the window, and the Motion event is sent to the target window.
4. Summary of Motion event distribution process
Combined with the Android input system (ii) IMS startup process and input event processing and the Android input system (iii) InputReader processing type and InputDispatcher distribution process, the Motion event distribution process can be summarized and simplified as the following figure.
- The Motion event is processed by the InputReader in the InputReaderThread thread. After processing, it determines whether to wake up the InputDispatcherThread. If it needs to wake up the InputDispatcherThread, The InputDispatcher is continuously used to distribute Motion events in the thread loop of the InputDispatcherThread.
- The Motion event is filtered by InputFilter. If false is returned, the Motion event is ignored.
- The data structure of InputReader after processing Motion events is NotifyMotionArgs. In the notifyMotion function of InputDispatcher, Construct a MotionEntry object with the event parameter information in NotifyMotionArgs. This MotionEntry object is added to the end of InputDispatcher’s mInboundQueue.
- If mInboundQueue is not empty, fetch EventEntry from the mInboundQueue header and assign it to mPendingEvent.
- According to the value of mPendingEvent, events are discarded.
- Call InputDispatcher findTouchedWindowTargetsLocked function, find the target for Motion event in mWindowHandles window list window, and generated inputTarget for this window.
- Get a Connection according to inputTarget and rely on Connection to send input events to the target window.
Here is a simple summary of the Motion event distribution process, and Motion events similar to the key event, you need to read the source code.
Afterword.
There’s a lot more to be said about the input system, such as how inputChannel communicates with Windows between processes and how InputDispatcher gets feedback from Windows, which will be covered in a future article in this series.
Thanks for Understanding Android In Depth volume 3
Share big front-end, Java, cross-platform technology, focus on career development and industry trends.
