Learn advanced usage of gesture processing from WebKit source - gesture recognition in WKWebView

More wonderful articles, welcome to pay attention to the author’s wechat public number: code worker notes.

Background knowledge of gesture processing in iOS

In iOS system, the screen click event mainly goes through the following steps from start to end [1] :

The user clicks on the screen to generate a hardware touch event
The operating system wraps hardware touch events as IOHIDEvent and sends the IOHIDEvent to Springboard, which then sends the IOHIDEvent to the currently open App process
After the App process receives the event, it wakes up the main thread Runloop (Source1), triggers the Source0 callback in the Source1 callback, and wraps IOHIDEvent into a UIEvent object. Send to the top level UIWindow (-[UIApplication sendEvent:])
UIWindow calls the hitTest method on each UITouch instance in UIEvent to find its hitTestView, and during the hitTest recursive call, Record the final hitTestView and its gestureRecognizers (gestureRecognizers)
- AddSubView calls hitTest in reverse order (addSubView calls hitTest first)
- The default is to recursively search its child view based on the return value of the pointInside method
Send each UITouch object to its gestureRecognizers and hitTestView (calling their touchesBegin method)
- The gestureRecognizer that successfully identified will monopolize the relevant touch, and all other gestureRecognizer and hitTestView will receive the touchsCancelled callback, and no more events for this touch will be received
  - A special case is that the gestureRecognizer on the parent view of the default UIControl (UISlider, UISwitch, etc.) control has a lower priority than UIControl itself
  - Also need to cooperate with related gestureRecognizer conflict resolution methods (such as canBePreventedByGestureRecognizer:, canPreventGestureRecognizer:) the specific implementation to use
- If the hitTestView instance does not respond to touchesBegin or other methods, the responder chain continues to find the nextResponder to call
- If a method such as touchesBegin is implemented, calling the [super touchesBegin] method in it passes the touch event up the responder chain

Two, WebKit applications

1. Expand the response area or range of touch gestures with hitTest

You can extend the area or scope of the click response by overloading the -hittest :withEvent: method and adding hitTest calls to non-current View child views. Note in implementation:

When you manually call the hitTest:withEvent: method, you need to transfer the coordinate point to the target View’s coordinate system
Call [super hitTest:withEvent:] to recursively hitTest child View (default logic)

//WKContentView
- (UIView *)hitTest:(CGPoint)point withEvent:(UIEvent *)event
{
    _interactionViewsContainerView / / here is at the same level with WKContentView in the View hierarchy (with a WKWebView as part of the View), to its View here call hitTest method, can reach to extend the clickable area of effect
    for (UIView *subView in [_interactionViewsContainerView.get() subviews]) {
        UIView *hitView = [subView hitTest:[subView convertPoint:point fromView:self] withEvent:event];
        if (hitView) {
            returnhitView; }}...// The default hitTest logic, recursively traverses the child View
    UIView* hitView = [superhitTest:point withEvent:event]; .return hitView;
}
Copy the code

2. Limit the response area or range of touch gestures with hitTest

By overloading the hitTest:withEvent: method and defining which specific views to hitTest or return directly.

Note that [super hitTest:withEvent:] is not called in the following routine, that is, the default logic of not doing hitTest through the subview

See the comments in the code:

//WKCompositingView
- (UIView *)hitTest:(CGPoint)point withEvent:(UIEvent *)event
{
    / / get hitTestView
    return [self _web_findDescendantViewAtPoint:point withEvent:event];
}

// It can be implemented as follows:

//UIView(WKHitTesting)
- (UIView *)_web_findDescendantViewAtPoint:(CGPoint)point withEvent:(UIEvent *)event
{
    Vector<UIView *, 16> viewsAtPoint;
    
    // Collect only non-WKCompositingView types that meet conditions such as position and save them in viewsAtPoint
    WebKit::collectDescendantViewsAtPoint(viewsAtPoint, self, point, event); .// The collected views are filtered according to business logic to find their target views according to different categories
    for (auto *view : WTF::makeReversedRange(viewsAtPoint)) {
        // Do a recursive hitTest for this view
        if ([view conformsToProtocol:@protocol(WKNativelyInteractible)]) {
            //natively interactible
            CGPoint subviewPoint = [view convertPoint:point fromView:self];
            return [view hitTest:subviewPoint withEvent:event];
        }
        
        // Return the view directly, without recursively looking for the child view
        if ([view isKindOfClass:[WKChildScrollView class]]) {
            if (WebKit::isScrolledBy((WKChildScrollView *)view, viewsAtPoint.last())) {
                //child scroll view
                returnview; }}/ / same as above
        if ([view isKindOfClass:WebKit::scrollViewScrollIndicatorClass()] && [view.superview isKindOfClass:WKChildScrollView.class]) {
            if (WebKit::isScrolledBy((WKChildScrollView *)view.superview, viewsAtPoint.last())) {
                //scroll indicator of child scroll view
                returnview; }}//ignoring other views
    }
    return nil;
}
Copy the code

3. Use gesture conflict resolution mechanism to support gesture rules defined by CSS touch-Action

The touch-action attribute in the CSS is used to set how the touch screen user manipulates the element area, mainly including the following values (see [2] for details) :

  /* Keyword values */
  touch-action: auto;
  touch-action: none;
  touch-action: pan-x;
  touch-action: pan-left;
  touch-action: pan-right;
  touch-action: pan-y;
  touch-action: pan-up;
  touch-action: pan-down;
  touch-action: pinch-zoom;
  touch-action: manipulation;

  /* Global values */
  touch-action: inherit;
  touch-action: initial;
  touch-action: unset;
Copy the code

For example, if the touch-action attribute of a DOM element is set to None, the WebView does not allow the element to be swiped with touch gestures.

Implementation scheme:

In order to achieve the support for touch – action, defines a special WKTouchActionGestureRecognizer in its, Add it to the WKContentView as the last gestureRecognizer (see the event dispatch priority in Part 1)
WKTouchActionGestureRecognizer touchesBegin/touchesMoved/touchesEnded methods of implementation, are called directly the _updateState will gesture recognition status set to success. According to the logic for handling gesture collisions described in Part 1, other Gesturerecognizers and hittestViews will receive the touchesCancelled callback — thus preventing other gesture responses.
- Note: not all gestures are blocked, and which ones can be left unblocked depends on the implementation of several methods to resolve gesture conflicts described below
WKTouchActionGestureRecognizer canBePreventedByGestureRecognizer: The gestureRecognizer method returns NO, indicating that it can recognize success and continue receiving Touches even if the other gestureRecognizer has recognized success
WKTouchActionGestureRecognizer canPreventGestureRecognizer: method, according to the touch – the action set, release or prohibit WKWebView in several predefined gesture

/ / WKTouchActionGestureRecognizer (as the last gestureRecognizer plus on WKContentView) / / touchBegin namely sets recognized status, Cancelled - (void)touchesBegan:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event { [self _updateState]; } - (void)touchesMoved:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event { [self _updateState]; } - (void)touchesEnded:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event { [self _updateState]; } - (void)touchesCancelled:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event { [self _updateState]; } // This method sets gestureRecognizer to recognize the status of success, This will make other Gesturerecognizers or hittestViews stop receiving events (in conjunction with other gesture conflict resolution methods) - (void)_updateState {// We always want to be in a recognized state so that we may always prevent another gesture recognizer. [self setState:UIGestureRecognizerStateRecognized]; } // Even if other Gesturerecognizers have recognized success, The gestureRecognizer still identifiable - (BOOL) canBePreventedByGestureRecognizer: (preventingGestureRecognizer UIGestureRecognizer *) {/ /  This allows this gesture recognizer to persist, even if other gesture recognizers are recognized. return NO; } // If gestureRecognizer succeeds, Then the CSS rules release part of the touch - the action of other gestureRecognizers - canPreventGestureRecognizer: (UIGestureRecognizer (BOOL) *)preventedGestureRecognizer { ... / / here _touchActionDelegate figure out whether preventedGestureRecognizer its mounted corresponding gesture recognizer auto mayPan = [_touchActionDelegate gestureRecognizerMayPanWebView:preventedGestureRecognizer]; auto mayPinchToZoom = [_touchActionDelegate gestureRecognizerMayPinchToZoomWebView:preventedGestureRecognizer]; auto mayDoubleTapToZoom = [_touchActionDelegate gestureRecognizerMayDoubleTapToZoomWebView:preventedGestureRecognizer]; If (!) is not mounted by webKit. mayPan && ! mayPinchToZoom && ! mayDoubleTapToZoom) return NO; // Now that we've established that this gesture recognizer may yield an interaction that is preventable by the "touch-action" // CSS property we iterate over all active touches, check whether that touch matches the gesture recognizer, see if we have // any touch-action specified for it, and then check for each type of interaction whether the touch-action property has a // value that should prevent the interaction. auto* activeTouches = [_touchActionDelegate touchActionActiveTouches]; for (NSNumber *touchIdentifier in activeTouches) { auto iterator = _touchActionsByTouchIdentifier.find([touchIdentifier unsignedIntegerValue]); if (iterator ! = _touchActionsByTouchIdentifier.end() && [[activeTouches objectForKey:touchIdentifier].gestureRecognizers ContainsObject: preventedGestureRecognizer]) {/ / set the pan - x/pan - y/manipulation, // Panning is only allowed if "pan-x", "pan-y" or "manipulation" is specified. Additional work is needed to respect individual values, but this takes // care of the case where no panning is allowed. if (mayPan && ! iterator->value.containsAny({ WebCore::TouchAction::PanX, WebCore::TouchAction::PanY, WebCore::TouchAction::Manipulation })) return YES; Pinch-zoom /manipulation PinchToZoom is valid only when pinch-to-zoom is only allowed if "pinch-zoom" or "manipulation" is specified. If (mayPinchToZoom &&! iterator->value.containsAny({ WebCore::TouchAction::PinchZoom, WebCore::TouchAction::Manipulation })) return YES; // When none is set, // double-tap-to-zoom is only disallowed if "none" is specified. If (mayDoubleTapToZoom && iterator->value.contains(WebCore::TouchAction::None)) return YES; } } return NO; }Copy the code

4. Conflict resolution between multiple Gesturerecognizers

WKContentView contains a number of Gesturerecognizers [3]. To resolve conflicts between gesturerecognizers, you need to override the following methods and implement the business logic for conflict resolution, as described in the following code comments:

//WKContentView 

// Tool methods
//static inline bool isSamePair(UIGestureRecognizer *a, UIGestureRecognizer *b, UIGestureRecognizer *x, UIGestureRecognizer *y)
/ / {
// return (a == x && b == y) || (b == x && a == y);
/ /}

// This method specifies which Gesturerecognizers can recognize at the same time, rather than one and then issue touchesCancel to the other person
- (BOOL)gestureRecognizer:(UIGestureRecognizer *)gestureRecognizer shouldRecognizeSimultaneouslyWithGestureRecognizer:(UIGestureRecognizer*)otherGestureRecognizer
{
    ...
    
    / / WKDeferringGestureRecognizer and toucheEventGestureRecognizer is conflict-free
    for (WKDeferringGestureRecognizer *gesture in self._deferringGestureRecognizers) {
        //isSamePair is used to determine whether the first two of the four input parameters are the same binary as the last two
        if (isSamePair(gestureRecognizer, otherGestureRecognizer, _touchEventGestureRecognizer.get(), gesture))
            return YES; }.../ / between WKDeferringGestureRecognizer is without conflict
    if ([gestureRecognizer isKindOfClass:WKDeferringGestureRecognizer.class] && [otherGestureRecognizer isKindOfClass:WKDeferringGestureRecognizer.class])
        return YES;

    // The highlight gesture and the long press gesture do not conflict
    if (isSamePair(gestureRecognizer, otherGestureRecognizer, _highlightLongPressGestureRecognizer.get(), _longPressGestureRecognizer.get()))
        return YES;

#if HAVE(UIKIT_WITH_MOUSE_SUPPORT)
    // Multiple mouse gestures do not conflict
    if ([gestureRecognizer isKindOfClass:[WKMouseGestureRecognizer class]] || [otherGestureRecognizer isKindOfClass:[WKMouseGestureRecognizer class]])
        return YES;
#endif

#if PLATFORM(MACCATALYST)
    // The magnifying glass does not conflict with the long pressing of the text gesture
    if (isSamePair(gestureRecognizer, otherGestureRecognizer, [_textInteractionAssistant loupeGesture], [_textInteractionAssistant forcePressGesture]))
        return YES;

    // Click and magnifying glass gesture do not conflict
    if (isSamePair(gestureRecognizer, otherGestureRecognizer, _singleTapGestureRecognizer.get(), [_textInteractionAssistant loupeGesture]))
        return YES;

    // Find does not conflict with the long press gesture
    if (([gestureRecognizer isKindOfClass:[_UILookupGestureRecognizer class]] && [otherGestureRecognizer isKindOfClass:[UILongPressGestureRecognizer class]]) || ([otherGestureRecognizer isKindOfClass:[UILongPressGestureRecognizer class]] && [gestureRecognizer isKindOfClass:[_UILookupGestureRecognizer class]]))
        return YES;
#endif // PLATFORM(MACCATALYST)

    if (gestureRecognizer == _highlightLongPressGestureRecognizer.get() || otherGestureRecognizer == _highlightLongPressGestureRecognizer.get()) {
        auto forcePressGesture = [_textInteractionAssistant forcePressGesture];
        if (gestureRecognizer == forcePressGesture || otherGestureRecognizer == forcePressGesture)
            return YES;

        auto loupeGesture = [_textInteractionAssistant loupeGesture];
        // Magnifying glass gestures do not clash
        if (gestureRecognizer == loupeGesture || otherGestureRecognizer == loupeGesture)
            return YES;
            
        //1.5 click gestures do not conflict
        if ([gestureRecognizer isKindOfClass:tapAndAHalfRecognizerClass()] || [otherGestureRecognizer isKindOfClass:tapAndAHalfRecognizerClass()])
            return YES;
    }

    // The following logic notes are omitted. Interested readers can peruse the WebKit source code
    if (isSamePair(gestureRecognizer, otherGestureRecognizer, _singleTapGestureRecognizer.get(), [_textInteractionAssistant singleTapGesture]))
        return YES;

    if (isSamePair(gestureRecognizer, otherGestureRecognizer, _singleTapGestureRecognizer.get(), _nonBlockingDoubleTapGestureRecognizer.get()))
        return YES;

    if (isSamePair(gestureRecognizer, otherGestureRecognizer, _highlightLongPressGestureRecognizer.get(), _nonBlockingDoubleTapGestureRecognizer.get()))
        return YES;

    if (isSamePair(gestureRecognizer, otherGestureRecognizer, _highlightLongPressGestureRecognizer.get(), _previewSecondaryGestureRecognizer.get()))
        return YES;

    if (isSamePair(gestureRecognizer, otherGestureRecognizer, _highlightLongPressGestureRecognizer.get(), _previewGestureRecognizer.get()))
        return YES;

    if (isSamePair(gestureRecognizer, otherGestureRecognizer, _nonBlockingDoubleTapGestureRecognizer.get(), _doubleTapGestureRecognizerForDoubleClick.get()))
        return YES;

    if (isSamePair(gestureRecognizer, otherGestureRecognizer, _doubleTapGestureRecognizer.get(), _doubleTapGestureRecognizerForDoubleClick.get()))
        return YES;

#if ENABLE(IMAGE_EXTRACTION)
    if (gestureRecognizer == _imageExtractionGestureRecognizer || gestureRecognizer == _imageExtractionTimeoutGestureRecognizer)
        return YES;
#endif

    return NO;
}

GestureRecognizer recognizes the priority only if otherGestureRecognizer fails to recognize it
- (BOOL)gestureRecognizer:(UIGestureRecognizer *)gestureRecognizer shouldRequireFailureOfGestureRecognizer:(UIGestureRecognizer *)otherGestureRecognizer
{
    // Normal touch events have a lower priority than left and right navigation (page forward and back) gestures
    if (gestureRecognizer == _touchEventGestureRecognizer && [_webView _isNavigationSwipeGestureRecognizer:otherGestureRecognizer])
        return YES;

    / / for deferringGestureRecognizer, if it need to delay gestureRecognizer (in fact determined by the delegate deferringGestureRecognizer), is here to specify it as a high priority
    if ([otherGestureRecognizer isKindOfClass:WKDeferringGestureRecognizer.class])
        return [(WKDeferringGestureRecognizer *)otherGestureRecognizer shouldDeferGestureRecognizer:gestureRecognizer];

    return NO;
}

/ / specify the priority between gestureRecognizer for deferringGestureRecognizer, if it need to delay otherGestureRecognizer, specified here its priority
- (BOOL)gestureRecognizer:(UIGestureRecognizer *)gestureRecognizer shouldBeRequiredToFailByGestureRecognizer:(UIGestureRecognizer *)otherGestureRecognizer
{
    if ([gestureRecognizer isKindOfClass:WKDeferringGestureRecognizer.class])
        return [(WKDeferringGestureRecognizer *)gestureRecognizer shouldDeferGestureRecognizer:otherGestureRecognizer];

    return NO;
}
Copy the code

5. Use WKDeferringGestureRecognizer to delay other gestureRecognizer recognition

Delay other gestureRecognizer recognition mainly has several application scenarios:

If multiple Gesturerecognizers are attached to the same view, and the touch sequences recognized by the different recognizers contain a common prefix sequence, a delay is required to recognize the success of those recognizers. To ensure that all recognizers have a chance to be recognized.
Another scenario is that the front end supports disabling the default gesture (event.preventDefault()) in event handlers such as TouchStart, which would require suspending other default gesture recognition while handling events such as Touchstart on the Web.
If the user initiates a touch gesture during scrollView scrolling, the new gesture should not be delayed.

WKDeferringGestureRecognizer implementation delay other main mechanism of gestureRecognizer identification process as follows:

In WKDeferringGestureRecognizer touchesBegan/touchesEnded methods, asking whether it’s delegate to defer this event at this time. If you don’t need to defer, directly to the self. The state = UIGestureRecognizerStateFailed, at which point it affect other gesture recognition; If you need to defer, your gestureRecognizer does not change its gesture recognition state, and other gestureRecognizer recognition operations that rely on it to fail will be delayed.
The logic to determine whether to defer at touchesBegin is to see if the corresponding touch.view is a scrollView and if the scrollView is interacting (SPI: _isInterruptingDeceleration), if so, do not defer, or defer
When touchesEnd is reached, it determines whether the front-end TouchStart event (which blocks other gestures) is currently being processed, and if so, defer
CanBePreventedByGestureRecognizer returned directly NO, says it will not be forced to cancel due to other gestureRecognizer recognition success

//WKDeferringGestureRecognizer - (void)touchesBegan:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event { [super touchesBegan:touches withEvent:event]; // If the delegate thinks it needs to defer, return without setting the failed state, so that operations that rely on it to fail will be deferred. if ([_deferringGestureDelegate deferringGestureRecognizer:self shouldDeferGesturesAfterBeginningTouchesWithEvent:event])  return; self.state = UIGestureRecognizerStateFailed; } - (void)touchesEnded:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event { [super touchesEnded:touches withEvent:event]; if (self.state ! = UIGestureRecognizerStatePossible) return; if ([_deferringGestureDelegate deferringGestureRecognizer:self shouldDeferGesturesAfterEndingTouchesWithEvent:event]) return; self.state = UIGestureRecognizerStateFailed; } - (void)touchesCancelled:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event { [super touchesCancelled:touches withEvent:event]; self.state = UIGestureRecognizerStateFailed; } / / couldn't be canceled by other gestureRecognizer - canBePreventedByGestureRecognizer: (UIGestureRecognizer (BOOL) *)preventingGestureRecognizer { return NO; }Copy the code

In WKContentView’s gesture conflict handler, the following methods are called to resolve the gesture prefix sequence

/ / gestures conflict related method invocation chain: - [WKContentView gestureRecognizer: shouldRequireFailureOfGestureRecognizer:] call: - [WKDeferringGestureRecognizer shouldDeferGestureRecognizer] call: - [WKContentView deferringGestureRecognizer: shouldDeferOtherGestureRecognizer:] / / WKContentView / / judgment here DeferringGestureRecognizer whether need to delay gestureRecognizer - deferringGestureRecognizer: (WKDeferringGestureRecognizer (BOOL) *)deferringGestureRecognizer shouldDeferOtherGestureRecognizer:(UIGestureRecognizer *)gestureRecognizer { #if The ENABLE (IOS_TOUCH_EVENTS) / / page forward and backward gestures should not be delayed if ([_webView _isNavigationSwipeGestureRecognizer: gestureRecognizer]) return NO; Auto webView = _webView.getautoReleased (); auto webView = _webView.getautoReleased (); auto view = gestureRecognizer.view; BOOL gestureIsInstalledOnOrUnderWebView = NO; while (view) { if (view == webView) { gestureIsInstalledOnOrUnderWebView = YES; break; } view = view.superview; } // Gestures in non-webView tree should not be delayed if (! gestureIsInstalledOnOrUnderWebView) return NO; / / other deferringGestureRecognizer should not be delayed if ([gestureRecognizer isKindOfClass: WKDeferringGestureRecognizer. Class]) return NO; / / web touch gestures should not be delayed if (gestureRecognizer = = _touchEventGestureRecognizer) return NO; auto mayDelayResetOfContainingSubgraph = [&](UIGestureRecognizer *gesture) -> BOOL { #if USE(UICONTEXTMENU) && HAVE(LINK_PREVIEW) if (gesture == [_contextMenuInteraction gestureRecognizerForFailureRelationships]) return YES; #endif #if ENABLE(DRAG_SUPPORT) if (gesture.delegate == [_dragInteraction _initiationDriver]) return YES; # endif / / 1.5 clicks gestures should be delayed if ([gesture isKindOfClass: tapAndAHalfRecognizerClass ()]) return YES; // Magnifying glass gesture should be delayed if (gesture == [_textInteractionAssistant loupeGesture]) return YES; / / single point click the gestures should be delayed if ([gesture isKindOfClass: UITapGestureRecognizer. Class]) {UITapGestureRecognizer * tapGesture = (UITapGestureRecognizer *)gesture; return tapGesture.numberOfTapsRequired > 1 && tapGesture.numberOfTouchesRequired < 2; } return NO; }; / / double click, click the gestures should be delayed if (gestureRecognizer = = _doubleTapGestureRecognizer | | gestureRecognizer = = _singleTapGestureRecognizer) return deferringGestureRecognizer == _deferringGestureRecognizerForSyntheticTapGestures; if (mayDelayResetOfContainingSubgraph(gestureRecognizer)) return deferringGestureRecognizer == _deferringGestureRecognizerForDelayedResettableGestures; return deferringGestureRecognizer == _deferringGestureRecognizerForImmediatelyResettableGestures; #else UNUSED_PARAM(deferringGestureRecognizer); UNUSED_PARAM(gestureRecognizer); return NO; #endif }Copy the code

The resources

Incident response in [1] iOS: www.jianshu.com/p/c294d1bd9…
[2] CSS touch-action: developer.mozilla.org/zh-CN/docs/…
In [3] WKWebView gestureRecognizer specific role: blog.csdn.net/hursing/art…

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Learn advanced usage of gesture processing from WebKit source – gesture recognition in WKWebView

Background knowledge of gesture processing in iOS

Two, WebKit applications

1. Expand the response area or range of touch gestures with hitTest

2. Limit the response area or range of touch gestures with hitTest

3. Use gesture conflict resolution mechanism to support gesture rules defined by CSS touch-Action

4. Conflict resolution between multiple Gesturerecognizers

5. Use WKDeferringGestureRecognizer to delay other gestureRecognizer recognition

The resources

Learn advanced usage of gesture processing from WebKit source – gesture recognition in WKWebView

Background knowledge of gesture processing in iOS

Two, WebKit applications

1. Expand the response area or range of touch gestures with hitTest

2. Limit the response area or range of touch gestures with hitTest

3. Use gesture conflict resolution mechanism to support gesture rules defined by CSS touch-Action

4. Conflict resolution between multiple Gesturerecognizers

5. Use WKDeferringGestureRecognizer to delay other gestureRecognizer recognition

The resources

Related Posts

IOS basic principles explore —–Runloop

Access the wechat open source library Mars in SWIFT

SDWebImage link redirection problem