Lu Kang, Chen Chi 枻, Nie Shuai

At present, the new force of car building is becoming more and more popular, and the intelligent car has become the hot spot. Many mobile phone applications hope to expand the car airport scene. Cloud Music and its subsidiary Look Live have also made some exploration in the car and airport scene

Current vehicle development types and characteristics

Currently, there are three main vehicle-mounted access methods. The first one is extended access of mobile phone APP represented by Huawei Hicar; the second one is external OpenApi, which is developed by auto companies themselves for access; the last one is the most common vehicle-mounted independent APP access.

1. Extended access of mobile phone APP. Take Huawei HiCar as an example

This approach does not require the vehicle to provide an independent vehicle version of APK, but the mobile application access Hicar SDK, directly developed on the original project.

At present, the solutions adopted by many mobile phone manufacturers are based on the MediaSession framework of the Android system for template development. Mobile applications only need to prepare data according to the template provided by the manufacturer, and the specific UI display is completed by the vehicle and the device. Developers do not need to care about screen adaptation and UI style unity. The specific synchronization of broadcast control instructions is also accomplished through MediaSession framework.

In this access mode, you need to customize the Media Data Tree structure. Since the display of the ViewTree is rendered externally, we often have to use mediaId and Extras in the onPlayFromMediaId callback to get the media information from the on-board click to play. MediaId can be structured into a hierarchy such as TAB -> Page -> listId -> songId, so that we know which songs are being played from which page and which playlist. This is how the Universal Android Music Player Sample is implemented.

This vehicle-mounted access mode has the following characteristics

  • Easy access, directly developed on the basis of the original project, the basic capacity is ready-made, the delivery form is mobile phone APK, consistent with the original
  • For example, Hicar directly provides the ability of template-based development for different types of applications. Audio applications only need to focus on the preparation of audio data and the realization of playback services, and other tedious work. For example, HiCar can draw the vehicle-machine interface and ensure the compatibility of various resolutions, manage audio desktop cards and connect audio tasks
  • Easy to update, only need to update the application on the mobile phone to update the vehicle display logic, compared to update the vehicle application, the cost of guiding users is much lower
  • Scope limitation, that is, only with a specific platform binding, such as Hicar only support huawei mobile phones, and car machine access huawei Hicar system, at present, the domestic several mainstream handset makers are trying to push a similar ecological, auto makers in the momentum of the car was made in the Internet, also speeded up the introduction of these systems, but from the total amount, Still a small part of the engine

2. OpenAPI access

In this way, the server provides corresponding OpenAPI interface according to our service content. Manufacturers can design their own requirements and visual plans, and call different interfaces to obtain data and display according to different requirements. However, they generally need to pass our review before releasing. The development resources in this access mode are also provided by the manufacturer itself, bearing multiple system environments such as Linux and Android. Since this approach is not the focus of this article, it will not be discussed here.

This access mode has the following characteristics

  • Our input manpower cost is small, the main development cost is concentrated in the manufacturer’s side
  • Can be adapted to a variety of environments, not limited to a single vehicle system
  • The controllability is small, and the acquisition of data partly depends on the manufacturer. We can only get the number of interface calls, so it is easy to have disputes on issues related to settlement
  • Iteration is difficult and depends on the vendor’s own development resources

3. Independent APP access

It can be seen that there are still some key problems in the implementation method of OpenAPI mentioned above, so we generally prefer the access method of independent APP, which is more common at present and also the access method mainly described in this paper. This approach is similar to mobile app development, but with a few characteristics

  • The fragmentation of vehicle and machine systems is more serious than the mature mobile phone ecosystem (the majority of the share is in the head manufacturers). Many manufacturers develop their own vehicle and machine systems based on Android. For the dependence of party control, desktop widget, instrument display and other devices, manufacturers often provide their own access SDK. So channel subcontracting is imperative
  • Vehicle-machine interaction requires simplicity and focus. Voice operation is a great attraction for users
  • Test vehicle equipment is relatively lacking
  • The range of system versions is large. Currently, the devices I have contacted can be covered from Android 4.3 to Android 10
  • Performance is generally weak, so pay special attention to performance bottlenecks when developing

The project design

In view of the characteristics of vehicle-independent APP development mentioned above, we have carried out a series of explorations in channel subcontract, decoupling vehicle-machine dependence, voice operation access, resolution adaptation and other aspects. The following are some relevant schemes

1. Multi-channel access capability abstraction

As mentioned above, the vehicle-vehicle system is fragmented. There are generally two situations to realize the control of vehicle-vehicle, such as square control, desktop widget and instrument display

  1. The manufacturer’s manipulation implements the Android native MediaSession specification. In this case, we need to respond to the relevant KeyEvent and call the MediaSession API to update the status at various playback related times
  2. The vendor provides SDK access for the manipulation, in this case we will follow the vendor’s custom specification

Considering that it is better for the upper business code not to be aware of platform differences, it decides to make a layer of encapsulation and isolation for channel access capabilityAs shown in the figure above, the channel dependency capability is abstracted into the EnvironmentDependency interface, which is implemented by different channels depending on their respective VEHICLE SDK. The Mediasession specification implements a generic class separately. The business layer sees a channel-independent DependcyWrapper proxy instance and only needs to invoke the proxy’s corresponding method at each business processing time, avoiding the business layer’s channel-related code. The response capability of party control is abstracted as EventCallback interface. After business implementation, the corresponding Dependcy instance is injected and triggered timely. To solve the problem of channel packaging, productFlavors solution of AGP is adopted. Different channels contain different source folders to isolate SDK dependencies.

FlavorDimensions "channel" productFlavors {// Dimensions "channel" buildConfigField("String", "channel", "" "" xp)} / / byd byd {dimension "channel" buildConfigField (" String ", "channel", "" byd" ")}... }Copy the code

2. Design and implementation of voice control

To do voice control, first consider the following questions

  1. Do you implement it yourself or do you use vehicle-machine capabilities?

From the docking experience, it is not common for manufacturers to provide the open ability of vehicle-machine voice at present. Even if some manufacturers do, their access and customization process is complicated and requires a long period. Therefore, it is a more reasonable choice for the application to integrate third-party SDK. However, for some manufacturers who need to support the voice assistant built in the car, we also need to provide corresponding solutions. 2. How does voice control evoke? (Can you provide other shortcut entrance except page click?) In order to realize the voice assistant evoked by specific phrases, the voice recognition SDK is required to listen for a long time in the application life cycle, which always preemption the mic focus, leading to the failure of the voice assistant built in the vehicle system (some vehicles have realized multiple microphone array, In other words, the system uses a separate MIC channel for sound collection, but this kind of vehicle is very few). Therefore, the short sentence evoking scheme is not feasible. Then, can we make use of party control? Party control can provide the confirmation key response generally, if the business itself does not need to confirm key (such as application for live business, do not need to pause, recovery) can be used directly to confirm key arouse the voice assistant, if need be, also can design a click on the way to raise (such as long as or double-click, this can be done by the time interval to determine key event in the business layer do), and, of course, The corresponding bootstrap also needs to be kept up, such as floating layer plus voice bootstrap when the user enters for the first time. How to map from speech recognition text to corresponding operation? The most convenient way is definitely for the client to directly judge the text matching, such as switching to the next live broadcast after recognizing the “next song”. However, this method has low fault tolerance, and the user will fail to make a slight adjustment of the statement. It is more reasonable to add the semantic recognition link after the voice-to-text link, and the process is as follows

Solve the basic problem, then to consider a more perfect voice assistant complete interaction processes, assistant to raise, will first enter the state and prompt voice support the type of operation, then the user input, if the input overtime will be prompted to assistant will close, after the normal input request parsing, to obtain the results after some operation will directly closed panel, Some operations will display the results directly in the panel and return to the query state, or will be prompted and return to the query state if it cannot be resolved. Therefore, the whole process is better abstracted into a state machine on the client side

  1. Car if you need different docking machine’s own voice assistant, involving control related instructions and callback to broadcast information out of the more common interface to realize, for common commands, such as play, pause, a song, the song, the collection, search on demand, need encapsulation into separate methods, such as car machine app to register a different server, The realization of the client is processed by the same client, and the results of the client processing can be returned to the server for display. The advantage of this is that the part of the connection with the vehicle and the machine is completely handed over to the server for processing, and the client only needs to perform corresponding operations according to the instructions delivered. The first half is decoupled. The second half is reusable

3. Multi-resolution adaptation

In the front-facing visual interaction design, considering the driving scene, the common operation area should be placed near the driver’s side as far as possible, and the interaction process should be as simple as possible, and the page jump level should not be too many. In addition to the mainstream horizontal layout, BYD, Xiaopeng and other car screens will also have a vertical screen. Common screen adaptation schemes include smallestWidth adaptation, modifying the headline DisplayMetrics#density scheme, using percentage layout, etc. Combined with the actual situation of the project, we suggest that most of the layout should adopt flow layout. Only change the direction of recyclerView in the layout can adapt to the switch of vertical and horizontal screens, and the card layout should be flat as far as possible. ConstraintLayout properties such as Guideline, layout_constraintHeight_percent, etc., are useful for implementing percentage layouts. If you have a screen with strange proportions and you can’t use flow layout, You can consider using the SW qualifier to ask visual students to provide layout adjustment strategies for a small number of special screens. In the development of a visual adaptation,, of course, our first reaction is to let all vendors may involve car machine equipment, but this is not realistic, from our experience, docking test car machine is quite short, some manufacturers even temporarily unable to provide the car machine, only provide documents, let us adapter again after their internal test. In this case, we can only simulate devices with different resolutions. Adb shell wm size command is the solution. It accepts parameters in the format of total length pixel value x total width pixel value and can be adjusted to the corresponding aspect ratio after running. During the test process, it only needs to run commands with different parameters on the same device to achieve simulation with different resolutions.

Performance optimization

As mentioned above, the overall performance of the mobile phone is far behind that of the mobile phone. At first, on the one hand, due to factors such as historical burden and component reuse, on the other hand also tend to ignore the performance related issues when writing code that makes the app running on the car’s very bad experience, slow, slow start installation, caton lost frame performance problems, such as obvious is exposed, so we made a series of targeted optimization

1. Reduce the package size

Reducing package size involves both code and resources, and the general approach is as follows:

  • Image compression
  • Resources to confuse
  • Reduce Dex

2. Reduce the number of processes

Multi-process operation needs to occupy more system resources. On devices with poor performance, the multi-process operation mode of single APP will bring more pressure to the DEVICE CPU and memory

Reduce the number of threads

Similar to processes, too many threads have a high overhead cost of switching frequently during startup, and the main thread takes less time to execute

4. IO optimization

Excessive FILE I/OS also slow down the startup process, minimizing unnecessary file reads and writes

5. Reduce the number of hops of your Activity

In order to display the interface faster or perform a specific function, it is best to reduce the number of jump levels of the Activity in the startup process. Each additional Activity adds hundreds of milliseconds. When requesting some interfaces, the request timing should also be taken into consideration, whether parallel requests can be preceded or combined to reduce the RT of interfaces

6. Optimize layout levels to reduce overdrawing

The following is an example of performance optimization. In the cooperation with a certain automobile manufacturer, the feedback of the manufacturer was that the playback speed was very slow from cold start to start, which lasted nearly 8s. There was no problem in our test on the mobile phone. However, due to the limitation of the performance of the vehicle and the machine, we mainly made the following optimization after several rounds of communication and joint adjustment

  • The package volume was significantly reduced, and a large number of useless business codes were deleted. The package volume was reduced by about 80%. As the package volume was greatly reduced, the number of dex to be decompressed during the startup process was also reduced, and fewer classes were loaded, and the speed was increased by several seconds
  • Combine the playback process with the main process, change the multi-process to single process, and remove AIDL. Remove several file reads and writes before and after AIDL communication, reducing about 2s time consuming
  • Combine LoadingActivity and home page Activity into one, reducing the number of activities that start a link and reducing the time taken by hundreds of milliseconds
  • Combining multiple interfaces into one reduces network requests by about 200 milliseconds

Finally, the time was compressed to 3s. In the early stage, we focused on optimizing the time-consuming part and achieved obvious results. In the later stage, we analyzed the startup log, cut a little details and finally passed the acceptance of the manufacturer. Before starting optimization, it is necessary to quantify the specific indicators and clarify the goals before starting. Using data to measure the optimization effect can make the optimization process more smooth

Guidelines on pit

In the process of vehicle development, I also encountered some problems that were not common in the development of mobile applications before, which were deeply impressed and shared here

1. Pre-installed RN pages are not working

In the vehicle-mounted scenario, the frequency of users actively downloading and updating APP is much lower than that of mobile phones, so preloading is a very important means of distribution. However, when we finally negotiated with a channel about preloading, we found a strange problem, all pages realized by RN would cause the crash of the application. Libjsexcutor. so, an RN dependent js parsing library, failed to load. It was found that RN’s SO library was loaded by SoLoader, a tool of Facebook (official documents said that it was mainly used to comply with the dependency problem of SO loading of the following version 4.3), while other business SO in the application were working normally. So I guessed that SoLoader would have problems in the application preinstall scenario, so I revisited and focused on the SoLoader related logsThe figure above shows RN load logs on the problem channel, and the figure below shows RN load logs in a normal scenarioIt can be seen that the difference between the two is that on the problem channel, the so search path marked in red is not added (this path is actually the soft link of the SO path after the application is installed), while on the normal channel, THE RN-related SO is found on this path and loaded. Following this idea, check the source code of SoLoader. The following logic is foundThat is, after judging that the current application is a system application, the default SO path of app will not be added to the search path, resulting in the failure of rN-related libraries loaded with Soloader. After locating the cause, I carefully examined the source code related to Soloader loading SO. Found its provides setSystemLoadLibraryWrapper Settings interface, the system can be defined by the upper application scenario how to load depend on the so, so we just set the scene with the application of the original so loading method can solve the problem, as shown in the following code

SoLoader.setSystemLoadLibraryWrapper {
    ReLinker.loadLibrary(context, it)
}
Copy the code

2. Strange problems on the test equipment

  1. After the test vehicle of a certain channel was connected to the company’s wifi, it could not access the network all the time. When I communicated with the manufacturer, they told me that it was the first time to provide the test vehicle to the outside and there was no problem for internal use, so I had to position myself. Considering the high probability of network environment, iptables tool is used to check the network rules of vehicle and machine (Iptables is an application software running in user space, which manages the processing and forwarding of network data packets by controlling the Linux kernel Netfilter module. The detailed flow of data packets is shown in the figure below. Rules can be added in each link to intercept)

It is speculated that the test vehicle is only for internal use of the manufacturer. In order to prevent problems after outflow, the network environment is identified. Once the data packet is found on the internal network of the company other than the manufacturer, the packet will be discarded

iptables -F
iptables -X
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
Copy the code
  1. Some interface errors were found in the development process of a certain channel. After a closer look, it was found that the error interface is HTTPS protocol (the development stage is still in the test environment, most of the interfaces are HTTP protocol), and the error content in the ADB log is roughly as follows
javax.net.ssl.SSLHandshakeException: com.android.org.bouncycastle.jce.exception.ExtCertPathValidatorException: 
Could not validate certificate: Certificate not valid until Wed Dec 16 
09:00:05 GMT+08:00 2015 (compared to Sun Oct 12 16:20:03 GMT+08:00 1980)
Copy the code

Seems to be the time and the validity of the certificate in, check the system time found that really wrong, originally the car machine will reset the system after each startup time, which contains the SSL client check check the validity of the certificate, then adjust the system time and solve the problem The visible and test car opportunity because of some special set leads to some strange development problems, However, the good news is that these test vehicles are often already root, so the command permissions are large enough for further analysis.

Experience beyond technology

I participated in the whole process of vehicle application from startup to launch. Besides technology, I also had some other experiences

  • Car factory project management is quite different from Internet products. Its style is rigorous and meticulous, seeking stability and not fast. Without the concept of Internet rapid iteration, it is often difficult to accept the practice of putting some problems online first and then iterating on fix, so its test cycle is usually long and there are many problem feedback rounds. Feedback issues can come from a variety of perspectives (product design, content operation, technical points), and the application side needs to be prepared and patient to deal with them.
  • Preliminary communication with factory, will align good delivery standards, such as adaptation range of demand, the application of performance indicators, etc., avoid back and forth for delivery standard not unified communication and rework, according to the situation of our project is to set their own standard baseline, usually by Monkey and performance test automation to ensure the stability of the app
  • Currently each depot app access to the overall process can’t say that is very perfect, there lack documents, test car machine lack, the instability of the simulator, the slower the response of the feedback problems, which requires the application side do homework early, the dependency to comb as soon as possible, and vendor timely communication, predict risk, behind with access to more and more widely used, There should be an improvement in the construction of the manufacturer section.

summary

This paper introduces the current situation of vehicle development, sharing some design ideas and typical problems encountered in the development process, hoping to be helpful to everyone’s application car!

This article is published from netease Cloud Music big front end team, the article is prohibited to be reproduced in any form without authorization. Grp.music – Fe (at) Corp.Netease.com We recruit front-end, iOS and Android all year long. If you are ready to change your job and you like cloud music, join us!