In this article, Demo synchronizes Github

preface

Why I do a social, live, picture editing direction of iOS development suddenly want to learn input method development, all this has to start from I saw sogou input method recruitment JD talk about….

I see sogou input method recruitment wrote a: understand the reverse priority, at this time I have a question, do input method App development and reverse what relationship? Therefore, I want to know about the input method App development interest, also has this article

At the time of writing this preface, I had no idea how input methods were developed. When I thought of making an input method App, I had the following questions

  1. Why sogou input method App installed, the system keyboard Settings will appear sogou input method, and there is a full access option (UISwith), what is full access?
  2. Why after installing sogou input method App, after adjusting the input method in my own App, the third party sogou input method keyboard is adjusted
  3. What is the connection between sogou input method App and Sogou input method keyboard? How to relate?
  4. What is the relationship between third-party input methods and reverse? What can reverse knowledge bring to input method apps?

If you have any of the above questions, please bear with me (actually I don’t know if I can answer them when I write here… All available in the afternoon)

Contents of this article:

  1. Research on
  2. App implementation (simple implementation of an input method App that can be used in other apps)
  3. Input method App principle
  4. Input method App and reverse (personal guess, maybe Lu Xun himself did not think so)

1. Start the research

1.1 hit a shell

Because the idea of understanding the input method App originated from reverse, so HERE I first crack the shell and see what I find…

After smashing the Sogou input method shell and looking at the files inside the IPA, I didn’t see any difference from other apps. Is nothing but icon resources, infoPlist, mpa resources, Frameworks, Lottie json folder, open the full access. Mp4 tutorial, MJRefresh third-party libraries, such as/PlugIns/SogouAction appex and Mach – o.

Through IPA, I found two key points:

  1. Sogou engineers prefer to use PList to do configuration data storage (this I think very good, easy to manage, of course. Json is no problem)
  2. inpluginsUnder the path, I have oneSogouAction.appexFile,.appexUsually the iOSTo expand the ExtensionGenerated files, such as accessNotificationServiceIt also generatespush.appexfile

At this point, I hit a shell on baidu input method also for operation, found in the PlugIns folder, there are also BaiduInputMethod. Appex, NotificationContent. These two appex. Appex at the end of the file

At this point, according to my intuition, I think the key of the input method App is to add an Extension implementation similar to inputKeyboard

1.2 the class – the dump

A search for SogouAction in the header file still turns up nothing

1.3 pass Reveal

I simply looked at the UI architecture of Sogou and tried to find out the corresponding class through the Reveal. However, the result was not found. The keyboard UI was not shown in the Reveal at all (the keyboard is the system-level UI, so all the Reveal did not show). Think of our goal is to understand how to develop an input App. At this point we temporarily stop reverse, go directly to Baidu…

1.3 Use search engines

Baidu search to few things, mostly detection of the keyboard pop-up height, only to find the key three articles:

Keyboards and Input

IOS input method _ Developing system architecture

IOS Input Method Development (Swift)

Article 1 is an official Document of Apple, mainly talking about the API used in the construction of input method App. Article 2 mainly describes the App architecture and communication of input method App. Through this article, we probably know why Sogou iOS needs reverse experience. Article 3 is to build a simple input method App. I decided to follow article 3 to develop a simple App to understand its principle

2. The App

2.1 Create a project named InputApp (for practice, we use OC encoding, different from the link, I will create Swift version and upload Github later)

2.2 Creating a Keyboard Target

The extension Target is named CustomKeyboard

Activate “CustomKeyboard” scheme? Choose to activate

Notice at this point that by adding CustomKeyboard, a class KeyboardViewController is generated by default

!!!!!!!!! Pay attention to an issue that’s easy to overlook: Remember to change the lowest supported version of CustomKeyboard Target. If the lowest supported version is higher than the device version, Xcode will compile without error, but the Target will not run at runtime. Adding NotificationService has the same easy to ignore problem

At this time, we start the App, and you can see our custom keyboard in Settings – General – Keyboard – Add Keyboard, and also have the option to enable full access



After adding the keyboard, we will switch the keyboard to our self-defined keyboard. You can see the picture as you adjust the keyboard in any App


When our custom keyboard pops up, we can see that the keyboard has a button that generates code by default when Extension is added. We’re going to do a custom layout for the keyboard in the KeyboardViewController class

2.3 Keyboard UI Layout

I uploaded the source code to Github CCInput

For the UI layout of the keyboard, I take the numeric keyboard of sogou input method as an example. The code description is attached here. For details, please see the Demo code

KeyboardViewController Vc automatically created by the system when keyboard Extension is created, which inherits from UIInputViewController. The keyboard layout and logic processing are in this class

CCLeftTableView is a TabView that supports adding symbolic data sources

CCCenterView middle numeric keypad and bottom switch, digit 0, space function

CCRightView right side delete, period, @ symbol, line feed function

CCKeyboardModel Keyboard data source Model, Model has two properties, respectively

NSString *string is used to display the text of keyboard buttons. CCKeyboardAction keyboardAction is an enumerated type of click events that are processed by this property when the keyboard is clicked

Simple to write a runtime automatically get properties parse JSON into model method

The UI and data structure are the main ones. I won’t go into them here. If you are interested, please see the demo.

  1. You cannot change the default keyboard height with a simple setFrame
  2. You need to setNSLayoutConstraintCut in the viewDidLoad method setting is invalid
  3. Needs to be inviewDidAppearBefore setting the keyboard height

Stackoverflow: iOS 8 Custom Keyboard: Changing the Height

The Demo:

The code in the Demo is also attached

static CGFloat KEYBOARDHEIGHT = 256; @interface KeyboardViewController () <CCTopBarDelegate, CCLeftViewDelegate, CCCenterViewDelegate, @property (nonatomic, assign) NSLayoutConstraint *heightConstraint; @end @implementation KeyboardViewController - (void)prepareHeightConstraint { if (self.heightConstraint == nil) { UILabel *dummyView = [[UILabel alloc] initWithFrame:CGRectZero]; dummyView.translatesAutoresizingMaskIntoConstraints = NO; [self.view addSubview:dummyView]; self.heightConstraint = [NSLayoutConstraint constraintWithItem:self.view attribute:NSLayoutAttributeHeight RelatedBy: NSLayoutRelationEqual toItem: nil attribute: NSLayoutAttributeNotAnAttribute multiplier: 0.0 constant:KEYBOARDHEIGHT]; self.heightConstraint.priority = 750; [self.view addConstraint:self.heightConstraint]; } else { self.heightConstraint.constant = KEYBOARDHEIGHT; } } - (void)viewWillAppear:(BOOL)animated { [super viewWillAppear:animated]; [self prepareHeightConstraint]; UpdateViewConstraints - (void)updateViewConstraints {[super updateViewConstraints]; if (self.view.frame.size.width == 0 && self.view.frame.size.height == 0) { return; } [self prepareHeightConstraint]; }Copy the code

Methods involved in UIInputViewController:

  • advanceToNextInputModeSwitch to the next input method
  • dismissKeyboardExit the keyboardresignFirstResponder)
  • insertTextInsert text (tail)
  • deleteBackwardDeletes the last text in the input box

3. App principle of input method

3.1 Relationship between the main program and keyboard expansion

The third-party input method can be divided into InputApp (containing App) and Extension Keyboard, corresponding to InputApp and CustomKeyboard in Demo respectively. The main program can be seen on the desktop, and Host App refers to other apps using the input method.

  • The main application and keyboard Extension normally cannot share data (sandbox isolation)

  • When full access is enabled, the main program and keyboard Extension can share data (via App Groups)

  • Main application and Host App cannot share data (sandbox isolation)

  • Keyboard Extension and Host App can only share text data (via system UITextDocumentProxy)

3.2 Full Access

In the Settings – General – Keyboard, Apple has instructions for full access, excerpted below

Third-party keyboards provide another way to type keyboard data. These keyboards have access to all the data you type, including bank accounts, credit card numbers, street addresses and other personal and sensitive information. These keyboards also have access to adjacent text and data, information that could help improve autocorrect.

If you enable full Access, the developer is granted permission to access, collect, and transfer the data you type. In addition, if a third-party application that ships with the keyboard has your permission to access geolocation information, photos, or other personal data, the keyboard can collect and transfer that information to the keyboard developer’s server. If you disable “full access” for a third-party keyboard and then re-enable it, the keyboard developer may be able to access, collect, and transmit information typed during the period when network access is disabled.

If you do not enable full access, the developer cannot collect and transfer the data you type. Any unauthorized data collection or transmission without your permission is a violation of its Developer agreement. In addition, technical restrictions also play a role in preventing unauthorized access. Any attempt to undermine such restrictions is also a violation of the Developer agreement.

You can choose to disable third-party keyboards at any time. Open “Settings” and tap “Keyboard” to remove the keyboard from the keyboard list.

If you use a third party keyboard, you are subject to the terms, privacy policies and practices of the keyboard developer. Before using such keyboard apps and services, you should carefully read their terms, privacy policies and practices to understand how they use your data and other information.

To sum up, only with full access enabled can the input method App:

  • Access, collect, and transmit incoming data
  • Access and upload location, photos, personal data (private data such as address book)
  • If it is turned on and then off, the developer will still be able to transfer data during the disabled period
  • If not, developers cannot upload data from keyboard Extension due to Apple’s technical limitations (sandbox)

3.3 App Groups Data Sharing

Sharing Data with Your Containing App

Simply put, enabling App Groups is equivalent to creating a shared container to associate and share data between App’s Container and Extension’s Container

You can share data in the following ways

// Create and share access to an NSUserDefaults object
NSUserDefaults *mySharedDefaults = [[NSUserDefaults alloc] initWithSuiteName: @"com.example.domain.MyShareExtension"];
 
// Use the shared user defaults object to update the user's account
[mySharedDefaults setObject:theAccountName forKey:@"lastAccountName"];
Copy the code

4 input method App and reverse (personal guess, maybe Lu Xun himself did not think so)

As we know from 3.1, 3.2, and 3.3 above, data sharing between the main App and Extension must be enabled in the form of App Groups. For a keyboard App, it must hope that the App and Extension can realize data sharing to meet the following requirements:

  • Update network high-frequency hot words in real time for associative input
  • Shared Keyboard skin
  • Data such as vocabulary and address book frequently input by users are stored and uploaded to the server. When users change devices, the Extension can be synchronized and shared with the new device

However, due to the limitations of the sandbox, if the user does not enable full access, the above three requirements are not possible. In fact the above three requirements such users is very important to me, the pleasure of seamless switching equipment is unspeakable, such as when I input ma on the iPhone, I chose the lenovo vocabulary pony code in the code, and then when I use on the MAC sogou input method input ma, same MAC also associate it with the pony of code code, This will make my input much more efficient

In this case, reverse technology is used. Apple has sandbox isolation, and reverse has sandbox escape.

Sandbox escape simply means cross-process access to data sharing data. In this case, even if the user does not enable full access, the main App can access Extension data to fulfill the above three requirements.

However, the current sandbox escape technology can only be implemented on jailbroken devices. Maybe one day someone will be able to implement sandbox escape on unjailbroken devices. That will be a great time for reverse developers.