New Machine Learning features for Android

Below is a text summary of the video content for your reference

I. Background knowledge

Device side machine learning has grown rapidly in recent years, Google I/O 2021 What’s New in Android Machine This part of Learning focuses on the advantages of machine Learning on the device side, common application scenarios, some technical challenges in deployment, and Google’s solutions to these problems.

Second, share content

2.1 Development and advantages of device-side machine learning

Over the past decade, machine learning has made great strides in vision, speech processing and the ability to perform language tasks.

Compared to cloud deployment, device side machine learning has many advantages:

  • Processing data on local devices reduces radio usage and thus power consumption
  • Eliminate round-trips and improve latency
  • Reduce exposure to private data
  • It can be used offline as well as online

Common applications: video conference background blur, real-time voice captioning, real-time pose detection using ML Kit, offline translation, handwriting recognition, etc. Other contents can be accessed at G.Co/on-device-ML

ML is growing rapidly on Android devices, with over 730 million devices including hardware acceleration, and ML Kit has over 500 million monthly active users

2.2 Some challenges of device side machine learning

Deploying these applications on Android presents some common challenges

  • Limitations on inclusion size
  • Performance differences, stability and accuracy between devices
  • Limited number of common machine learning apis due to high device compatibility (supported by different versions)

Can there be a set of components to solve these problems?

2.3 Solutions

Provide a fully integrated machine learning inference stack that can be updated

  • No need to bundle and integrate machine learning-related components
  • Can provide a variety of hardware acceleration for performance optimization
  • A unified API for all Android versions, with regular updates from Google

TensorFlow Lite for Android

  • Preinstall TensorFlow Lite for Android devices using Google Play. You do not need to bundle the installation, reduce the package size, and automatically update the package using Google Play
  • TensorFlow Lite integrates hardware acceleration to increase CPU performance by 50% by adding metadata to the model to select the best performing back-end for the model.

TensorFlow Lite has a powerful CPU back end and uses Android-based hardware acceleration

I wanted the model to run on a GPU, which is very complicated on Android

  • Performance, stability, and accuracy all vary depending on the chip’s combination of GPU drivers
  • Experiments need to be set up to obtain data from different equipment for training

At present, some solutions have been provided. Starting from TFLite 2.3, Android version has built-in GPU acceleration support list, which only needs to be instantiated and used safely through the GPU proxy class to confirm the supported hardware calls, but there is no way to measure whether the acceleration effect is achieved after using the GPU

Automatic acceleration is provided to take the model into account and, with additional metadata, to check that the model is working properly and is performing better than the CPU when using the accelerator. The automatic support list applies not only to cpus and Gpus, but also to other DSPS and Npus

Android neural network API

With the addition of DSP and NPU hardware units in new ANdroid models, NNAPI can guarantee access to these underlying hardware units on different devices

But since Android 10 NNAPI has been integrated into the Android OS, updates are slow, new models run only on new devices and require developers to automatically adapt to different devices

Android 12 starts with NNAPI being removed from the OS, updated via Google Play, and new hardware drivers that can be more practical. Different versions of Android – the hardware uses the same NNAPI, and Qualcomm will be a partner for updatable NNAPI products

Third, summary

3.1 Content summary

Device side machine learning, a new Android feature

  1. Google Play-enabled phones will automatically integrate TensorFlow Lite on the device without having to package it into the app.
  2. The machine learning framework supports automatic hardware acceleration. By adding meta-data to models, the optimal performance of models such as CPU, GPU, and NPU can be automatically compared and selected.
  3. NNAPI is separated from Android OS side and updated through Google Play, and corresponding hardware drivers can be actually updated, so as to provide unified compatible NNAPI on different Android versions and hardware devices.

3.2 the key QA

Q: How does TensorFlow Lite integrate directly on the device side?

A:www.infoq.com/news/2021/0… . As a base API library, it updates common components through the Google Play Service

Q: How is automatic hardware acceleration implemented?

A: The simple understanding is that the running status of the model is automatically collected and scheduled during each running process without specifying hardware Backend, and the best solution is found.

Q: What exactly is NNAPI?

A: The neural network library was integrated at the level of Android OS before, and then it was separately separated and updated through Google Play Services. In addition, we are cooperating with hardware manufacturers to unify corresponding hardware compatibility and driver update.