At Google I/O 2018, Google released the ML Kit. With ML Kit, even developers with no machine learning background can quickly develop projects based on machine learning. Calling ML Kit’s API is just as easy as calling any other mobile native SDK. Sweat_smile: This is really exciting news for me, who has tried machine learning many times and is still trying. However, can Google’s ML Kit be widely used in its own apps? What are the advantages and disadvantages? Especially for China’s special network environment, and what about it? This article will provide a brief overview of the ML Kit development experience, as well as an unprofessional measure of the tool’s strengths and weaknesses from the perspective of a machine learning novice.

By the way, Google is really smart about naming ML Kit, which will probably drive a lot of traffic to Google from search engines. Every year at WWDC, Apple likes to name their new SDKS in FooKit format, such as ARKit, MapKit, etc. However, Apple’s MACHINE learning SDK isn’t named after MLKit. Instead, it’s called Core ML (in line with one of Apple’s previous naming preferences). I’m sure a lot of people will mistakenly type “MLKit” in their search for Core ML, so Google will be able to guide their own SDK in the future.

Note: There are two main sample apps in this article:

  • MLKitDemo (github.com/imWildCat/M… , text recognition Hello World project)
  • MLKitExample (Firebase official example)

Development experience

For the average iOS developer, creating and running an ML Kit hello World project will take less than five minutes.

To create an iOS native OCR project (hereafter referred to as “MLKitDemo”), add the following three lines of code to your Podfile:

pod 'Firebase/Core'
pod 'Firebase/MLVision'
pod 'Firebase/MLVisionTextModel'Copy the code

Then, create your own app in the Firebase Console (possibly with a credit card attached), download the corresponding GoogleService- info.plist, and use it. Simple text recognition looks like this:

func performTextRecognition() { textDetector = vision.textDetector() let uiImage = UIImage(named: "demo_image") let visionImage = VisionImage(image: uiImage!) let visionMetaData = VisionImageMetadata() visionMetaData.orientation = .bottomRight visionImage.metadata = visionMetaData textDetector! .detect(in: visionImage) { (features, error) in guard error == nil, let features = features, ! features.isEmpty else { print("Error. You should also check the console for error messages.") // ... return } // Recognized and extracted text print("Detected text has: \(features.count) blocks") for block in features { print("\"\(block.text)\"") } // ... }}Copy the code

Depending on the project and taking up space

The following information is based on MLKitExample (the official Firebase example), not the Hello World project (MLKitDemo) above.

Although the APIS integrated with ML Kit are very simple, the SDK dependencies are quite large (source: pod Install) :

Installing Firebase (5.0.0) Installing FirebaseAnalytics (5.0.0) Installing FirebaseCore (5.0.0 FirebaseInstanceID (3.0.0) Installing FirebaseMLCommon (0.9.0) Installing FirebaseMLModelInterpreter (0.9.0) Installing FirebaseMLVision (0.9.0) Installing FirebaseMLVisionBarcodeModel (0.9.0) Installing FirebaseMLVisionFaceModel (0.9.0) Installing FirebaseMLVisionLabelModel (0.9.0) Installing FirebaseMLVisionTextModel (0.9.0) Installing GTMSessionFetcher (1.1.15) Installing GoogleAPIClientForREST (1.3.4) Installing GoogleMobileVision (1.3.0) Installing GoogleToolboxForMac (2.1.4) Installing Protobuf (3.5.0) Installing TensorFlowLite (0.1.7) Installing nanopb (0.3.8)Copy the code

Pre-trained Models are relatively large (source: NCDU) :

635.2 MiB [##########] /GoogleMobileVision 29.5 MiB [] /TensorFlowLite 18.2 MiB [] /FirebaseAnalytics 4.1 MiB [] / FirebaseInstanceID 4.1 MiB/FirebaseMLVision 2.7 MiB [] [] / FirebaseMLModelInterpreter 1.6 MiB/Protobuf 556.0 [] KiB [] /GoogleAPIClientForREST 484.0kib [] /GTMSessionFetcher 292.0kib [] /Target Support Files 284.0kib [] /FirebaseCore 232.0kib [] / Pods.xcodeProj 124.0Kib [] /nanopb 112.0Kib [] /GoogleToolboxForMac 80.0kib [] / FirebaseMLCommon 12.0 KiB/FirebaseMLVisionLabelModel 12.0 KiB [] [] 12.0 KiB/FirebaseMLVisionBarcodeModel [] / FirebaseMLVisionTextModel 12.0 KiB/FirebaseMLVisionFaceModel 12.0 KiB [] [] 8.0 KiB/Firebase [] Manifest. 0.0 the lock B [] /Headers 0.0b [] /Local PodspecsCopy the code

Pre-trained models size under GoogleMobileVision:

252.4 MiB [# # # # # # # # # #] 250.0 MiB/FaceDetector [# # # # # # # # #] / TextDetector 88.2 MiB / # # # 26.8 MiB/LabelDetector [#] /BarcodeDetector 17.7 MiB [] /DetectorCopy the code

Deployment and usage experience

Deploying ML Kit-based apps on iOS is simple and requires no additional operations, just installation.

App size

With pre-trained Models included, the app is larger than normal:

Experience with

Text Recognition

The accuracy of text recognition is also very high, even if the photos are a bit blurry. Here’s an example from the Economist:

However, the pre-trained Model only supports English text recognition. For Chinese or Chinese-English mixed text recognition, a custom model needs to be loaded.

Face Detection

Face recognition is naturally more accurate, this piece of all have done more mature, let alone Google. Take president Donald Trump, who is popular with the public:

Image Labelling

Again, a photo of President Trump was used and the object was identified with a high degree of confidence, but its location was not marked:

Internet use

For well-known reasons, Google’s services are not always available in China. I specifically looked at web usage using Little Snitch (apps that only have offline models) and found that there are still apis that call Google:

Play.googleapis.com is apparently pinging and inaccessible. So, unfortunately, the SDK doesn’t seem to be directly available in China right now.

conclusion

From TensorFlow to AutoML, from TensorFlow Lite toML Kit, it’s not hard to see Google’s extensive footprint in machine learning. To be fair, ML Kit is a good example of Google’s ease of use: providing rapid development kits while providing great flexibility through TensorFlow Lite. But inevitably, ML Kit is bundled with Firebase (Firebase app credentials need to be configured even for offline inference), and using Firebase in China is not an easy task. If the app’s primary audience is Chinese users, the next best thing is to use TensorFlow Lite to deploy the machine learning model on mobile.