This is my share at Droidcon Beijing 2016 and GDG Devfest 2016. Here is the text:

Slide 01 The topic I share today is bluetooth low Power on Android. The subject is relatively niche. For the past year or so, I have been developing bluetooth for low power. BLE development is a bit different from typical Android APP development. It requires access to hardware resources, involves some protocol-related content, and has little information on this topic. Today I’m going to share some low-power Bluetooth development practices from an Android developer’s perspective.

Slide 02 Today’s content to share, mainly includes the following parts: First of all, to do a simple introduction of Bluetooth and low power Bluetooth; Then introduce the low power Bluetooth support on Android. What low power Bluetooth applications can be developed on the Android platform? Then, tools to help us debug during development; Finally, summarize some of the so-called “best practices” of low-power Bluetooth development.

Slide 03 Before introducing bluetooth low power, we have to mention its superset bluetooth. Bluetooth penetration is quite high, I believe everyone knows it, the logo is like this (picture). To put it simply, Bluetooth is actually a near field communication technology, which was first developed by Ericsson. Bluetooth Bluetooth the name Bluetooth looks like a portmanteau of blue teeth, but it is not. The word Bluetooth is a nickname for the King of Denmark. The engineers who developed it were reading a book about the king and came up with the name.

Bluetooth SIG is the international organization responsible for the development and promotion of Bluetooth specifications. Do bluetooth related products, and it is indispensable to deal with. The Bluetooth Technology Alliance also owns the Bluetooth trademark, which requires their authorization to use.

Let’s go back to Bluetooth itself. What are the features of Bluetooth? First of all, near distance communication, the typical distance is less than 10 meters, the transmission speed up to 24 Mbps, support multiple connections, high security, very suitable for smart devices.

A brief introduction to bluetooth version evolution. In 1999, Bluetooth 1.0 was released, and later bluetooth 2.1 was the most widely used version. Today, most bluetooth products are in this version, which is also what we call classic Bluetooth. Bluetooth 3.0, also known as high-speed bluetooth, has a maximum transmission speed of 24Mbps. Bluetooth 4.0/4.1 introduces low power Bluetooth in this release. Bluetooth 5.0 will also be released soon, mainly for improvements in the direction of the Internet of Things.

Slide 04 Let’s take a look at Bluetooth Low Energy, or BLE for short. As the name suggests, its biggest feature is low power consumption, with some BLE devices running for a year or two on a button battery. There are some other advantages, such as low cost, fast connection, high security. In addition, low power consumption is not without its drawbacks. Low power means low transmission rate, and it is designed to transmit small amounts of data. In general, Bluetooth low power is ideal for wearable devices or the Internet of things.

Slide 05 Let’s take a look at the BLE protocol stack. As Android developers, we don’t have to understand every detail of the BLE protocol stack, so here’s an overview of the protocol architecture. As we all know, protocols are usually designed in layers. The BLE protocol stack is no exception. Let’s take a look at this diagram. The whole protocol stack is roughly divided into three parts, from bottom to top: Controller → Host → Applications.

Controller: It is the implementation of the bottom layer of the protocol stack, directly related to the hardware, generally directly integrated in the SoC, implemented by the chip vendor, including the physical layer and the link layer. Host: this is the upper implementation of the protocol stack, is the abstraction of the hardware, and the specific hardware and manufacturer is independent. Application layer: applications developed using apis provided by the Host layer.

Protocol stack inside the module is more, we will briefly introduce a few and our development related to a few simple introduction.

Slide 06 First, the physical layer. Bluetooth is working around 2.4GHz, which is the industrial, scientific and medical ISM band. You can see it works on the same frequency band as WiFi. Bluetooth divides the frequency band into 40 channels, 3 broadcast channels and 37 data channels, which communicate according to certain rules (GaUSS frequency shift keying GFSK).

There is an interface layer, called HCI for short, between the Host layer and Controller layer. Hosts and controllers interact with each other through HCI commands and events. The HCI layer is optional in the protocol stack. For example, it may not be available on some simple and small devices, but it certainly is available on all Android devices. This is the only way to interact with bluetooth’s upper application and chip. As we’ll see later, this layer of logs is a great way to analyze and debug problems.

Slide 07 In the Host part, the protocol structure is more complex, including logical link control and adaptation layer, security management module and so on. Let’s focus on the attribute protocol, or ATT for short, which is the basis of BLE communication. ATT encapsulates data and exposes it as “attributes”. The server provides “attributes” and the client obtains “attributes”. ATT is designed specifically for Bluetooth low power, with a very simple structure and very short data length.

Slide 08 Next, let’s take a look at GATT, the generic properties profile, which builds on ATT and further encapsulates it, defining how data interacts and what it means. This is a concept we came across directly when we were doing BLE development. GATT defines three concepts in terms of hierarchy: services, characteristics, and descriptors. Their inclusion relationship is shown in the figure on the right: a Service contains several Characteristic, and a Characteristic can contain several descriptors. Characteristic defines values and operations. Characteristic operations: read, write, notification, etc. BLE communication is Characteristic read and write or subscribe notifications. The outermost layer, the Profile Profile, combines several related services together to form a Profile. A Profile defines an actual application scenario.

Slide 09 One more thing here is that services, Characteristic, and descriptors are uniquely identified with UUID. And we’ll talk about that later. So what is UUID? UUID is a globally unique identifier. It is a value of 128 bits. To facilitate identification and reading, the UUID is generally in the following format: hexadecimal identifier 8-4-4-12.

There are several specifications for UUID, and to avoid conflicts, they are generally not defined manually. For example, uuid.randomuuid () is provided in Android to generate a randomUUID. We also saw that the UUID was a bit too long, which would not be convenient in the case of the very limited data length in low-power Bluetooth, so bluetooth used the so-called 16-bit or 32-bit UUID again. There is no 16bit or 32bit UUID per se. The Bluetooth SIG defines a basic UUID (Bluetooth Base UUID) in the following form. The 16 bit UUID corresponds to a 128 bit UUID. For example, there are only 65536 16-bit UUID’s, so 16-bit UUID’s cannot be used arbitrarily. SIG has predefined UUids, which you can buy if you want to add your own 16-bit UUID.

Slide 10 On top of that, there is the generic access control profile, also known as GAP, which, as its name suggests, defines the flow of BLE’s entire communication process, such as broadcast, scan, connection, and so on. It also defines the role of devices participating in communication, as well as their respective functions, such as Broadcaster of broadcast data, Observer receiving broadcast, Peripheral being connected and Central initiating connection. As you can see, the device roles participating in the interaction are not peers. The detailed interaction process will be discussed later.

Finally, there is the application layer, which is the low-power Bluetooth application developed using the API provided by Host. So far, we have taken a look at the BLE protocol stack, which gives us some theoretical basis for developing BLE.

Slide 11 Next, let’s take a look at low power Bluetooth support on Android.

Slide 12 Low power Bluetooth has only been supported since Android 4.3 Jelly Bean, also known as API 18. This supports BLE Central mode, which is what we said in the GAP above, where Android devices can only connect to other devices as Central devices. Peripheral mode has only been supported since Android 5.0.

The BLE API of the Android SDK is under Android.bluetooth. *, and some apis under Android.Bluetooth. le* are also introduced in Android 5.0.

For example, introduces some concepts of the GATT definition in front, has the corresponding class, such as BluetoothGatt/BluetoothGattService/BluetoothGattCharacteristic, etc. With this introduction, it’s easy to understand what these classes do.

In addition, to use Bluetooth in the APP, you need to apply for Bluetooth-related permissions in the Manifest. On Android 6.0 or later, you need to apply for location permission. Why is that? Because BLE does have the ability to locate, which we’ll talk about later.

Slide 13 Let’s take a look at the architecture of bluetooth implementation on Android. Let’s look at the diagram on the right, which is a very classic hierarchy on Android. The bottom part of the hardware can be connected to Android AOSP through hardware Abstraction Layer (HAL) interface and Java access interface through JNI, using the specific implementation of each vendor. Bluetooth services run in the com.Android.Bluetooth process and eventually provide apis to clients, namely apps, through Binder mechanisms.

A little more about the Bluetooth protocol stack. Android 4.2 used to implement BlueZ using the old protocol stack. 4.2 has been replaced with BlueDroid, which is jointly developed by Google and Broadcom and used exclusively on the Android platform. As a new implementation, BlueDroid’s functions are not perfect. We can see that BLE was not supported until 4.3, and peripheral mode was not supported until 5.0. So far, the functions are not perfect. There are also some bugs.

Slide 14 This section describes the process of BLE operation in Android. APP initiates a BLE operation and then understands and returns the operation result. The operation is encapsulated as a message and then placed in a message queue in the protocol stack, where a separate thread gets the message for processing, much like the well-known Looper and Handler mechanisms.

Because you’re using messaging, you have to know which client to notify when you call back? Before the client initiates a request, it first registers the client with the protocol stack. After successful registration, it returns a clientIf, which is an integer and is a handle of the client in the protocol stack. Subsequent operations of the client only need to carry this Handle.

At the end of the operation, there is usually an explicit stop operation to free the previously requested clientIf and resources. If not freed correctly, not only will memory leak, but all subsequent BLE operations may become unusable. Because this clientIf is limited, there are only 32 in the Bluetooth protocol stack, and it is shared by all apps on Android. When these resources run out, they can only be restored by killing the corresponding APP or restarting bluetooth.

Slide 15 So that’s it for BLE implementation on Android. Now let’s talk about what you can do with BLE on Android, and what applications you can develop.

Slide 16 BLE applications can be divided into two broad categories: unconnected and connected.

Non-connection-based, such applications are BLE dependent broadcasts, also known as beacons. The party who delivers radio is called Broadcaster and the party who listens to radio is called Observer.

Connection-based, which means sending and receiving data by establishing a GATT connection. There are also two roles, the one making the connection is called the Central device, and the connected device is called the Peripheral.

Slide 17 Let’s start with the disconnected, broadcast Beacon based application. Its network topology is as follows. As we know, Broadcaster is unidirectional. Broadcaster broadcasts outwards and listeners receive nearby broadcasts, which generally forms a unidirectional star shape. A network can have multiple peripherals, as well as multiple listeners (animation).

Completely based on broadcast applications, there is the famous iBeacon, which is a function defined by Apple Based on BLE broadcast, which can achieve advertising push and indoor positioning. This also shows that APP using BLE requires location permissions. Some time ago, mobile QQ can “find lost children” project, in fact, is also this principle, in children’s products implanted can send specific broadcast equipment, mobile phone scan, according to the geographical location of the mobile phone, can roughly determine the scope.

There are also devices that can perform both roles, sending and receiving broadcasts. A device receives a broadcast, processes it, and then forwards it, creating a two-way Mesh called a Bluetooth Mesh. Such a network would be able to transmit information from one point of the network to any point, as long as there were enough dense nodes in the space. This can be applied to the Internet of things and smart home systems.

In the introduction to the physical layer of the protocol stack, we know that the broadcast is only broadcast on the three broadcast channels 37, 38 and 39, and the listener also listens on these three channels. As we have mentioned before, Bluetooth communication is frequency hopping. Only when both devices jump to the same frequency at a certain moment can they receive the broadcast. This kind of data transmission has low efficiency and limited data volume, which is not suitable for large-scale data transmission.

Slide 18 Next, let’s talk about broadcast data, which is broadcast packets. When we talk about broadcast Data, there are two parts: Advertising Data and Scan Response Data. In general, the broadcast party broadcasts Advertising Data to the air ata certain interval. When a listening device listens to the broadcast Data, it sends a Scan Response Request to Request the broadcast party to send the Scan Response Data. The length of both parts of the data is fixed at 31 bytes. In Android, the system concatenates the two data and returns a 62-byte array.

The structure of the broadcast packet is shown in this figure (animation). The broadcast packet contains a small AD structure one by one. Each AD structure is a complete data, and its structure is: the first byte represents the length N, followed by n bytes of data. In the data section, the first byte represents the data type, which is the meaning of the data, and the n-1 byte represents the actual data. For example, 0x08 is the name of the device, and the following data is the UTF-8 encoding of the device name.

The broadcast data can be manually parsed. In Android 5.0, ScanRecord is also available to help you parse the broadcast data.

What data types can you have in a broadcast? The device connection property, which identifies the BLE mode supported by the device, is required. The device name, the key GATT Service that the device contains, or service data, vendor-defined data, etc.

Here we can also see that broadcast data only has a maximum of 62 bytes, so broadcast data space is very valuable per direct. What kind of data do you put into the broadcast? The principle is to broadcast some data that can express the identity of the device, and some necessary data that needs to be exposed. Bluetooth 5.0 is expected to dramatically expand the capacity of broadcast data to 512 bytes.

In addition, by the way, wireless communication basically has the concept of signal strength — RSSI, the UNIT of RSSI is dB, from which the distance can be roughly estimated. However, this is very unreliable on Android devices. RSSI values fluctuate greatly depending on the environment and the Angle of the phone.

Slide 19 So let’s see what that scan looks like. For example, we use an APP to scan and scan a device, and these contents are resolved from broadcast data, such as device name, device type, GATT service data, manufacturer customized data, etc. The raw data looks like this (animation). Here is a 62-byte hexadecimal identifier. In the table below, each row is an AD Structure.

Slide 20 Let’s take a look at how Android as a receiver receives broadcast data and scans devices. The code is actually very simple, first create a LeScanCallback, used to receive the broadcast, callback reported data. The BluetoothAdapter startLeScan is used to start the scan. When the scan needs to be stopped, stopLeScan is used to stop it.

Let me take a closer look at this callback function, onLeScan, which has the BluetoothDevice parameter, which represents the scanned device, and the key is the MAC address information of the device. Then there is RSSI, which represents the signal strength of the scanned device, and then scanRecord, which is the broadcast data we introduced earlier, which is 62 bytes long. It is worth mentioning that all of BLE’s callbacks are not in the main thread.

There are a few points to note here. StopLeScan must be adopted after scanning is not required, and the LeScanCallback passed in start and stop must be the same, because LeScanCallback is the identity of our client. Otherwise, clientIf will not release as we said earlier. In Android development, we often use anonymous inner classes for arguments, so don’t do that here.

In Android 5.0, bluetooths Canner, a new scanning API, is available, which provides finer control over scanning.

In addition to this approach, you can also use a classic Bluetooth scan, BluetoothAdapter’s startDiscovery(), and then receive the broadcast via BroadcastReceiver. This method is not recommended if you are just doing BLE development, it is a very heavy operation with very little flexibility.

Slide 21 Let’s take a look at the scanning workflow. First, the APP initiates a scan request and sends the request to the Bluetooth chip through the Bluetooth Service. The Bluetooth chip starts scanning, detects the device, and reports it via a callback. As we know, the scanning is really implemented in BT chip. As long as the APP sends the request, the Android system can sleep, which is often referred to as AP (Application Processor). After the device is scanned, The underlying BP (Baseband Processor) wakes up the upper AP and performs a callback notification to the APP, as shown in the red box in our figure. There is a problem here. With the gradual increase of BLE devices around us and frequent scanning of devices, the system will be frequently woken up or even unable to sleep, resulting in serious power consumption.

In order to avoid this problem, the problem of power consumption. We need to use as little scanning as possible. Even if a scan is required, we want to report as few devices as possible. Here, you can use the new interface provided by Android 5.0 to set ScanFilter to filter by certain rules. Only devices that meet our rules are detected will be reported, or delay the reporting can be set to reduce The Times of waking up the system.

Here’s a summary of some recommendations for scanning. 1. First, use new apis as much as possible to make them more powerful. 2. Scan as little as possible, because after all, scanning is a heavy operation that consumes power and slows down THE BLE connection speed; 3. When scanning, try to set ScanFilter to scan only those devices you are interested in, not the whole thing; 4. Use API correctly, especially stop scanning reasonably to prevent resource leakage.

Slide 22 Let’s take a look at the application of Android as Broadcaster. Starting with Android 5.0, Android devices can send BLE broadcasts just like peripherals. At this time, Android devices can exchange data through BLE, or discover each other’s devices, such as NFC like simple information exchange applications, there is still a lot of imagination.

The code implemented in Android is as follows. From the previous introduction, we know that there are two kinds of broadcast packages: Advertising Data and Scan Response Data, we set these two packages here, and BluetoothLeAdvertier startAdvertising. So the thing to notice here is the same thing as before, Start, Stop.

Slide 23 That’s it for connectionless applications, and we’ll move on to connection-based BLE applications. Its network topology is shown below. We see that there are two roles involved in communication: the Central and the Peripheral. A central device can be connected to multiple peripherals, and a peripheral can only be connected to one central device. Communication between central devices and peripherals is bidirectional. You can see that this is a typical star configuration. In fact, a central device can be connected to a limited number of devices at the same time, generally up to 7 peripherals.

Slide 24 BLE connections are established through GAP to negotiate and create connections. The Central device initiates a connection, and peripherals receive connection requests and negotiate connection parameters.

We introduced GATT earlier, and the core of GATT is services, Characteristic, and Descriptor. Each BLE peripheral, depending on its function, exposes services and so on. In fact, the most important thing is to acquire Characteristic in Service. Characteristic can be read, written, and notified when it changes, thus realizing two-way communication.

Slide 25 Let’s take a quick look at the structure of the GATT through an example. In the image on the right, we connect a Mi band 2, where each line in the list is a Service, such as Generic Access, Device Information, etc., as well as a string of UUID without a name. What’s going on here? As mentioned earlier, the Bluetooth Alliance has pre-defined a number of services, which are 16bit UUID. These services have standard meanings and purposes. For example, the Generic Access service describes device connection-related properties. There is also the Immediate Alert service, which indicates that the device supports standard Immediate Alert functions, such as xiaomi wristband as an Alert notification device and Hear Rate as a heart Rate service. The labels that directly display UUID are customized by the manufacturer, including 128 bit long ones and 16 bit short ones. If the labels with short ones are purchased, it can be seen that the manufacturer has purchased FEE0. For details about how to customize a Service, only the vendor knows the meanings of its attributes.

There are also two services that are going to be expanded, and they’re going to be Characteristic, and some of them are going to be Descriptor. The following labels indicate the permissions for this feature. For example, the W label indicates that the feature is writable. The UUID of Character is similar. If we write 1 in Alert Level, the bracelet should be able to send a warning. Read the Hear Rate directly to get the heart Rate. In addition, there are also manufacturers to define, the content of the data inside, only the manufacturer knows.

The official definition of services and Characteristic are all defined on the official website of Bluetooth, which can be viewed through the connection in KeyNote.

Slide 26 This is how to use GATT connections in Android. As you can see, the Android SDK-related classes correspond to the GATT concepts. As mentioned earlier, BLE operations on Android are asynchronous and BluetoothGattCallback needs to be created for callbacks. This class is very important and all callbacks for GATT operations are in this class.

BluetoothGatt is obtained through the connectGatt() method of the BluetoothDevice, which represents a connection, and when it is out of use, remember to close() to release resources.

Slide 27 Let’s take a look at the flow of GATT operations on Android. In the figure on the right, APP is our application, and the arrow from left to right on the Bluetooth server is the request initiated by APP, and the arrow from right to left is the callback. We see that all operations are done asynchronously. GattConnect is used to initiate a connection, onConnectionStateChange() is notified if the connection is successful, and if so, discoverService() is executed. This step is to discover all GATT services on the device. If discovered, onServiceDiscovered() callback is used to confirm the connection. Then by BluetoothGatt getService () to obtain BluetoothGattService, achieve BluetoothGattCharacteristic etc, then then to read and write the Characteristic.

This is the ideal situation. It is possible that when you attempt to establish a GATT connection without a callback and initiate a GATT connection at the same time, there is a high probability that the connection will fail. It is best not to make multiple connection requests at the same time.

In addition, in a GATT connection, you cannot initiate another Characteristic read/write operation (animation) until the last one is returned, otherwise the next one will fail. So for a connection, requests must be executed sequentially, with one request completed before the next.

We expect request and response to come in pairs, but the reality is that sometimes a request can be sent without a callback. We said that no new request can be made before the previous callback, so are we just waiting here? To address these issues, a solution is recommended below.

Slide 28 To meet the requirement of having only one connection request, we can create a connection management class in which only one device is being connected at a time. Since the connection requires multiple interactions, using a state machine is the most appropriate solution. State transitions are initiated based on callbacks, and a threshold is set for the time spent in a state beyond which state transitions occur even without callbacks. Put all connection requests in a queue and wait for the previous connection to succeed or fail before making the next connection request.

Next, for established GATT connections, all operations are asynchronous and need to be performed sequentially. To ensure this logic, it is very difficult to write. Imagine a scenario where you need to read and write multiple Characteristic characters in a row. The solution is to encapsulate asynchronous calls into synchronous calls using Wait /notify. It’s very easy to write and not prone to status errors.

Slide 29 After Android 5.0, we can also use Android devices as peripherals, connected to central devices. At this point, Android devices can truly use BLE for two-way communication, and there are many application scenarios. Specific implementation and Central basic corresponding, understand the PRINCIPLE of GATT, the implementation here is very simple. I will not go into details here.

Slide 30 We’ve covered all four roles of an Android device as a BLE application: listener, broadcaster, hub, and peripherals. It also introduces some possible bugs encountered during development, but how do we Debug BLE related problems during actual development? Let’s look at some Debug tools that can help us.

Slide 31 First of all, we can use some BLE test apps to verify some basic functions. There are many similar apps on the APP market, and nRF Connect is particularly recommended. This APP is like an API demo. It does all the things that Android API provides. It is of high quality and very easy to use.

In addition, viewing the Log is the most common method. Logcat is most familiar to Android developers. The com.android.bluetooth process log can be viewed in Logcat to learn about the status of the Java layer.

Further down, we can look at the log of the protocol stack. Bluetooth protocol stack log is not printed by default, we can open it in developer’s choice. (Animation). As shown in the figure, turn on these two switches:

Start with the following switch, turn on this switch, in Logcat will output BlueDroid Log, through these logs can check BlueDroid implementation of some problems.

Sometimes, sometimes knowing the protocol stack log may not be enough, turn on the first switch, the system will grab the HCI layer log file, usually in /sdcard/bt_snoop.log. At the HCI level, we talked about the way that the Host and Controller interact. This log is very similar to the network capture we do on a network card. In fact, the log file can also be opened through WireShark, the network packet capture software, as shown in the figure, you can see all HCI commands and events, and the corresponding data. This log can be used to troubleshoot bluetooth chip problems.

In fact, you can also go further, you can check whether the Bluetooth chip is working properly, that is, through some devices directly grab bluetooth air packets. This requires the purchase of specific hardware and software to work with.

Slide 32 Finally, let’s summarize some of the so-called best practices for BLE development in Android.

Slide 33 Finally, let’s summarize some of the so-called best practices for BLE development in Android.

Slide 34 First, we should do as little as possible. Because bluetooth has limited hardware resources, fewer requests to the chip are better. In addition, BlueDroid is still young, and the less pressure from the top, the better. Under the condition of ensuring business, we can try to follow the following principles: as little as possible connection number, as short as possible connection time, as little as possible scanning. As for ClientIf, if you want to scan or connect in multiple modules in APP, you should merge it as much as possible. Broadcast as little data as possible, make as few parallel requests as possible, and do as little work in callbacks as possible.

Slide 35 In addition, we need to do as much as possible. Although Bluetooth is a unified standard, there is still a gap between different manufacturers, for example, MTK and Qualcomm platform is somewhat different. In addition, bluetooth this piece, different ROM manufacturers will have some modifications, need to do some tests on different ROM, finally connected peripherals, implementation is not quite the same, but also need to do as much testing as possible.

Bluetooth is still developing rapidly. Both the specification of Bluetooth and the support of Bluetooth on Android platform provide us developers with more space and possibilities. We look forward to the emergence of more useful and fun applications for Android BLE.

Slide 36 In the end, there is no way to cover everything, and there is a need for more insight in the development process. Here are some resources:

  1. This is bluetooth 4.1 protocol document, this is the most authoritative information;
  2. This is the official Bluetooth website, all kinds of information about Bluetooth can be found here;
  3. Here are some articles I wrote about Bluetooth;
  4. This is the introduction to low power Bluetooth development on the Android developer website.