On November 23, byte technique salon | Flutter special technology End of the back art space in Beijing. We are joined by Huihui Yuan, the Flutter architect from Bytedance’s Mobile Platform Department, Justin McCandless, the Google Flutter team engineer, Li Mengyun, a senior Flutter engineer from Bytedance’s Mobile Platform Department, and Wang Shubin, a senior technical expert from Alibaba.
The following is a summary of the theme, Cross-platform Technology Trends and Bytedance Architecture Practices, shared by Huihui Yuan, Flutter architect of the Bytedance Mobile platform.
Outline of speech:
To introduce myself
Good afternoon! The topic I want to share with you today is Cross-platform Technology Trends and bytedance architecture Practices.
I joined Bytedance this year and am currently in charge of the optimization of the Flutter architecture in the Mobile Platform Department. Before that, I worked in Xiaomi for 3 years for the underlying optimization and research and development of android mobile phone system. In my spare time, I like to write blogs. I have been writing for four years since 2016, producing more than 200 original technical articles. During this period, I was invited to attend some Technical conferences related to Android in the industry.
In the face of an endless stream of new technologies
Let me ask you a question before I share with you: When new technologies are emerging, do you want to stay with the old ones or try new ones? You don’t have to think, what’s your intuition going to be?
But when faced with this question, a lot of people think: Should I try something new? In this example, a lot of people think, am I going to end up with nothing if I keep changing technology? I don’t want to try a new technique if I’m going to stick to it and eventually get what I want and get results. But there’s a myth:
First, if you do long enough time in one area, three years, five years, ten years, so you may learn, tired at that time you will enter the technology, may not willing to challenge new things, are likely to feel growth space is limited, if you are willing to jump out of the comfort zone technology, this time to take a look at the surrounding new technologies, It will broaden your knowledge. For me, I worked on the Android driver layer before, and then I worked on the Android Framework layer above. When I came to Bytedance, I started doing The Flutter. It was a kind of deep dive as I was constantly choosing new directions. Get your technology stack through.
Second, if you choose to learn more new techniques, you can see a lot of diamonds below, you want to choose to find the biggest diamond, how to find? Look at the trend. A good trend may not help you find the biggest diamond, but it certainly helps you find the bigger diamond.
Third, how do you find a new idea? Do you start from scratch every time you work like the person on the left? This is also a myth. Experienced people know that when you are deeply ploughed in a field, learning a new field is not starting from zero. To a certain extent, you will develop horizontally and get through, rather than returning to zero. Your learning cost will not be that long, because the technology is interlinked.
So in our pursuit of technology, how do we find bigger diamonds? It’s looking at technology trends. How to harvest more diamonds? Don’t stick to one direction, of course, it is not recommended that you switch frequently, but you can gain in multiple areas. How to find diamonds more efficiently? Our technology is interlinked, when you are learning new technology, you need to connect the original technology, get through, integrate your responsibility to the six pulse.
To address these issues, let’s take mobile cross-platform as an example. First, let’s look at trends in mobile technology; Look at the core technologies of the Flutter engine and how they relate to the existing technologies. Finally, I will introduce the technical progress and landing practice of bytedance in the new field of Flutter, and see how to achieve fast and deep development in the new field.
1. Mobile cross-platform technology trends
Why do we do mobile cross-platform technology?
-
Mobile Internet has developed for more than 10 years, from the traditional PC terminal era to the mobile era. In the mobile era, competition is fierce, and major mobile Internet companies are scrambling to grab the market. How to improve the efficiency of research and development, product rapid iteration, rapid trial and error has become a very important factor.
-
At the same time, also look at focusing on the r&d efficiency at the same time can make a compare the application of high performance, our r&d efficiency and performance is often on both ends of the scale, the performance good development efficiency may be poor, development efficiency good performance may be poor, this needs thinking about how to put the development efficiency and performance of both achieve a better balance.
-
Moreover, with so many cross-platform technologies, can an application be developed in one language with a consistent experience across multiple platforms? UI little sister designed this version of xiaolongnu, research and development little brother developed another xiaolongnu, to the Android platform into a variety of xiaolongnu, found that the UI looks different. Can we achieve an efficient, consistent experience across multiple interfaces?
-
Also, can I break through the channels to update my app quickly? Not only is this cross-platform technology, but Native itself is also concerned with this issue.
The above are the most concerned factors of the pros and cons of cross-platform technical solutions.
How has cross-end technology evolved over the years?
In 2011, we used WebView technology to do cross-end. This cross-end had great limitations, such as poor performance and limited functions, and it could not do complex scenes. In 2015, Facebook released React Native, which is based on JS language. Through the middle layer conversion, the final rendering is tuned to the Native layer, and the performance is relatively better. Next Ali launched Weex, also through JS language to native rendering. Google released Flutter in 2018. It has its own engine, its own rendering mechanism, and its performance is quite good. So that’s sort of the evolution.
Cross-terminal technology type
Cross-end technologies fall into several categories:
-
Native render: use various techniques to adjust the middle layer conversion, final render, Android or Android, iOS or iOS.
-
WebView: Render by tuning WebView technology, our small program is based on this piece.
-
Self-rendering: You can write your own self-rendering technique without a system, which is the focus of today’s lecture – Flutter.
No matter how cross-terminal technology evolves, there are three main categories.
What technical advantages does the Flutter have?
-
Efficient: Cross-platform, Dart language can run on more than one side at a time, which is undoubtedly efficient.
-
High performance: self rendering, do not rely on system native rendering, use your own.
-
High consistency: When a set of code is written, it behaves consistently across multiple ends.
-
Dynamic: It’s not appropriate to talk too much about this, but the engine itself is trying native engines.
Let’s look at high consistency, here’s a Demo written by Flutter, which is pretty much the same on Android and iOS. The little sister of UI tends to design UI in the hope that multiple experiences will be consistent, and it can be highly consistent.
Let’s talk about high performance, why is it often called high performance? Take Android as an example. The Android Native Framework APP is tuned to the familiar Android Framework, Flutter is tuned to Skia, Skia is tuned to the GPU.
How do you do native render for the general frame on the right? It has a conversion layer, and by converting, you end up with Android rendering, iOS rendering, iOS rendering, and you have an extra layer in the middle.
With the Flutter, the midtune is a Dart Framework, and then you tune it to Skia, and you use the Flutter platform and it’s very close to the native. The Flutter is a very high cap technology, if you optimize it well enough, it can match the native. Of course, Flutter may have fallen short as a new baby, but as it develops, it can do even better.
Industry companies can directly do Flutter, BAT, bytedance and more or less have a presence in their business, which is a trend.
In addition, Google has an in-house system called Fusicha, which also uses the Flutter and Dart on top. What about the future of this system? As 5G and IoT markets continue to evolve, it’s likely that the system will work well as well. If it works well, you may have an extra chance to overtake in the corner if you get the Flutter early.
Decode the principle of the Flutter engine
Technical Architecture of Flutter
Let’s look at the Flutter technology stack. Up here is the APP written in Dart, and down here is the DartFramework, which has the body of Android and iOS, a lot of animations and so on. And then we go down to the engine, which has messages, PlatformChannel, Dart VM, etc., and then the engine layer to the platform, so we’re looking at the platform, Android, Web, and that’s one of the architectures that we have in general.
Engine Genesis
But we have to look at a system from a dynamic perspective. What is a dynamic perspective? How was the Flutter created? The life cycle of it and how does it end up?
The image above is still an Android example, there are applications and activities, and apps written with Dart will eventually be connected to the native, the Application of Flutter and the Activity of the Flutter. In the Flutter Application, the code that Dart wrote will generate a product, load it up, and pull up our engine during our Flutter Activity.
In the engine layer, the four core threads of the Flutter are platform thread, UI thread, GPU thread, and IO thread. Each of them has their own responsibilities. The platform thread is often called the main thread for Android and iOS, and the UI thread is an independent thread facing the main thread of Android. GPU threads are those that run on the CPU and do Skia work. IO threads, such as picture decoding, codec, mainly do IO related work. Since there are several threads here, it must involve communication between threads, so let’s go back to technology communication, if you are familiar with Android and iOS, you know how threads communicate, so let’s look at the difference between the two in a moment.
There is the Dart virtual machine on the right, and there is the concept of the Isolate, which is the concept of the Flutter, as you can see from the name, “isolated”, what does that mean? Two pieces of memory are not isolated or unshared. If you look at the top right corner, you’re in the Dart layer, and you’re going to have UI drawing.
Let’s talk about these four pieces in turn.
Flutter compilation product
The first image is to see what the code that Flutter writes in Dart looks like in the end. The Dart code is immediately compiled with the front-end compiler, the green one on the left is Android, and the blue one on the right is iOS, depending on the platform. On the left is an APK, and on the right is the Runner APP on our iOS. The bottom engine code generated by compilation is different on Android and iOS, and the two parts are pieced together to become APK for different platforms. It turns out that code written in Dart will be compiled on different platforms for different applications, and this is the result of the final compilation. The Flutter is able to load this product up and run.
Thread communication
The second figure, talking about thread communication, again using Android, you’re familiar with Android, how do we communicate? There’s a lot of messages in there, so basically it’s putting a message on a message queue, it’s an Android technology. If we look at the right Flutter, it’s amazing how the technology is connected. The Flutter still has a Looper thread, which reuse the original Native thread for the main thread. An independent Looper is created for the other three threads. The difference is that it has two message queues, which still runs messages through Task Runner, just like android Handle. Passing a PostTask is like putting a message on a message queue. While Furture and asynchronous methods are often used in Dart, the core is to send a message using our Task Runner.
We need to see the similarities and differences between technologies. What are the differences? The difference is we have a MessageQueue here, we have two queues here, what does it do? The micro Task is dealt with first and then the normal Task is dealt with after the micro Task is dealt with. So it’s called Task in Flutter and Message in Android. The technology is wonderfully similar. You don’t have to worry, it’s very expensive to learn a new technology, but once you’ve mastered the technology, it’s very cheap to learn a new technology, it’s very easy to understand.
The Dart vm
Look at the ISOLatets. There are many ISOLatets in the same process. Two ISOLatets cannot be shared, but they need to interact. How to interact? The Dart VIRTUAL machine also has a special Isolate that runs on the UI thread and the same thread as the Root Isolate. When two ISOLates want to communicate, they find a common accessible memory. Android hears a lot about the concept of “interprocess communication”. How does it eventually communicate? User mode processes cannot be shared, but kernel mode processes can be shared, so find a place to share the communication data in the kernel mode and put it in the queue of the other side, and then the other side can get it.
The same logic applies to Isolate. How do Isolate1 and Isolate2 communicate? Create a ReceivePort in Isolate2, call SendPort in Isolate1, and set the mapping table in PortMap. Each port corresponds to an Isolate MessageHandler. The Handler has two queues. One is a common message queue and the other is an OOB message with a high priority. Then, the event is encapsulated into a MessageTask and then thrown into another Isolate. We usually create an Isolate, which contains a worker thread. When the worker thread is placed into a new Task, it will eventually execute the Task. Eventually the message will be parsed and you will see that the technologies are connected again.
Widget
And then we go up to the UI layer and there are widgets, and there are a lot of widgets in the whole Flutter. A Widget can be a line of text, an image, a color, all widgets. The two largest types are stateless widgets and stateful widgets. Stateless is a class of StatelessWidget that, once created, is immutable; The second type of StatefulWidget is one where the state can change. This is where state is different from Android, now that you have state, as you write more and more complicated applications you have state management, how you manage state.
Rendering principle
Create the Widget tree for each Widget, which is the Element’s configuration. If a change is made between the two widgets, it will make a difference, compare which part has changed, and only update the change to the Element, at the smallest granularity. The Render object is created when you build the Render in the Element, creating the Render process.
Render comes a signal, the UI thread updates the animation, the animation is finished to do a build, build the process to generate Render, and then the layout, drawing size and other work. When this is done, a Layer Tree is generated, also spanning two threads, UI and GPU threads. Once on the GPU thread, Skia is called to render.
Platform Channels
Why use platform channels? People often say that Flutter is a beautiful UI tool, but what if you really need Native ability sometimes? For example, to call camera features you need to write Native code, so The Flutter provides a Channel mechanism, the orange part of the code, to write android or iOS custom code for that platform, and there’s a mechanism in the middle that encapsulates it for you, so you can write Dart code and call it to Android or iOS code. This process is asynchronous.
You’ve seen the core technology of the Flutter engine, not only the UI layer, but also the engine layer mechanics. I mentioned many times in the process of point, the technology is interlinked, you in the field of a deep, new field still can continue to deep.
Byte skipping practices in the Flutter architecture
Bytedance has over 20 applications that use Flutter, headlines, watermelon videos, etc., and a lot of internal applications that use Flutter, it’s not a paper project, it’s a lot of businesses that use Flutter.
Flutter Architecture Practices
What is our mobile Platform department doing with the Flutter?
In order to support our applications, we first made a lot of efforts in the application framework layer in the engineering, with the container idea, how to make the business connect faster; Knowledge of hybrid engineering, enabling businesses to quickly access Flutter; Our state management, how to access efficiently in the case of mixed states; Also for our business better access, we have a lot of technology component development, providing a lot of rich libraries.
In addition, to monitor the Flutter, there are a number of high performance platforms available, with stability testing and so on.
The engine layer has also been heavily optimized, and although the Flutter is claimed to be high performance, it still needs to be optimized for business use. First of all, a lot of our engineers just started writing the Dart and didn’t know how to write the Flutter code more efficiently, so the business could be optimized, and second of all, there was a lot of optimization of the engine, and a lot of integration, and compilation, and dark tech, and we spent a lot of money in each area.
Containerized architecture
Let’s take a look at our containerization. For business support, there are many apps that provide containerization extension interfaces. For example, to adjust the picture, the picture to our protocol layer, will adjust a picture of the protocol, there is a default adaptation, there is a user can customize, you can not do any customization, we will help you provide common technology. Below with Android, iOS interaction can be ignored, writing code is very convenient, many libraries have, this is the idea of container, help you close the bottom difference, let you access more quickly.
Monitoring system
Looking at the monitoring system, we found that the engine itself has our monitoring, the TOP is the GPU thread, the bottom is the UI thread, this is a rough performance statistics method. Why rough? There are several deficiencies: first, performance is divided into UI and GPU threads, so we obviously want to see one parameter when we look at parameters; second, its algorithm is very rough, it counts the time of each frame, makes physical average, and finally normalizes the average time according to 60fps corresponding to 16.6ms.
Many companies in the industry have made improvements. How have they improved? The usual way to figure out how long the UI thread takes at the framework level is to look at how long it spans and how many frames it loses. What’s the problem with this approach? We can look at this diagram and understand the rendering process. When a signal comes, you post the message to the UI, Handle it, and then post the thing to our GPU thread. The drawbacks of this scenario are: First, there is a messaging process between the Post thread and the UI thread, and it is possible that the waiting time during the Post process is not taken into account. Second, what are the drawbacks if you just count UI threads? For example, the UI thread is very fast, but the GPU thread is very slow. The UI thread may finish in one or two milliseconds each time, but the GPU thread takes one or two hundred milliseconds each time. Although the performance is very good, the real experience shows that the card burst and the GPU can hardly brush. When the UI thread sends a message to the GPU thread, it can Post a maximum of two messages. At this time, the GPU thread still cannot process the message, so the UI thread will not Post the message, but the UI thread cannot reflect the message, so this is also a defect of the industry solution.
We have made a high-precision non-intrusive performance monitoring scheme:
-
First of all, we need to know how many frames to draw, so we’ll count how many signals you send, how many GPU threads you actually draw, and that’s the real experience. We are providing a set at the engine level.
-
Second, why noninvasive? Often performance monitoring will set a start when we start scrolling and an End when we End scrolling. We do not need to do this action. The access framework system will automatically identify the scene and automatically determine when to start and End, so it is high precision and non-invasive performance monitoring.
Multi-end integration practice
There have also been attempts in multi-terminal integration. Internally write an application that can run on iOS, Android and Web at the same time, and string some internal components together to make a multi-terminal integration attempt. The core of the multiplex integration is that we have a regular Flutter engine, and a Flutter Web on the right, calling Dart2JS, and that includes us on the Web.
Flutter Turbo
The Flutter Turbo has a performance improvement solution. How can we maximize our performance with limited resources? The core idea is that I need to find the core scene to do effective resource scheduling, for example, the user is sliding on the interface this scene is very important, can you control some resources effectively? For example, whether the message scheduler can prioritize the response to my first time, whether I can suppress GC in scenarios that are very important to the user, turn off the special effects that can be turned off, there are a series of means, in order to improve our performance, there is an internal Benchmark score, these solutions can bring improvement.
Image passthrough optimization scheme
How to upload existing pictures? So what you do in a regular Dart is you download an image off the network and you call Image.net Work, and then you load that image, and then you send the data to the Dart layer, you send a path and so on, and the Dart layer then calls the engine layer to decode it. The external texture is also very good, but it has some problems, instead of revamping this base, I made my own set of thoroughly existing schemes.
On the right is a diagram of how we modulate the engine in the Dart layer. When the engine loads an image, we load it inside and it generates a Bitmap. Although the engine is the Dart virtual machine, we share the Bitmap between the two. We convert it to the Pixel buffer, which also gives a big performance boost.
Start speed optimization
We started early, and it started very slowly, but we did a lot of changes in the engine layer, and it almost doubled the start speed.
Package volume optimization
Package volume is a point that people are very concerned about. Why do you care about package volume? Will determine whether or not the user is willing to download the app, so we did a lot of work on the package size, a lot of effort.
-
Do compilation optimization, how to adopt more efficient compilation, iOS uses O compilation, found that the package volume is further compressed.
-
The Flutter product has a data side and a code side. We compress the data side further and the packet size will be reduced.
-
In Skia, we did tailoring, cutting out functions that were less needed and things that were less necessary.
-
In the engine layer, we also did some tailoring of the three-party libraries.
-
At the instruction end we find that the instructions written with Flutter are a little longer than the instructions we use with OC code. We have made positive communication and feedback with Google on this issue, Google has spent a lot of energy, and we also follow up and promote Google to do this thing.
After a series of schemes, our package volume will be nearly 50% optimized.
These are some of the highlights of the many practices that bytedance has done in Flutter.
Four,
First, we share the technical trend of Flutter, which is a hotly sought after technology.
Second, I introduced the Flutter engine and found that the technology inside the engine is similar. The message communication mechanism is almost the same as the Handle mechanism on Android. But it has its differences, the technology has its differences.
Third, bytedance cloth a new field, found that we do a lot of technology deep ploughing. Going back to the original question, do you want to stick with a new technology, or do you want to try something new? It turns out that making new technology isn’t as hard as everyone thinks, and you can learn a lot along the way.
Fourth, since you’re picking trends, how do you know you’re picking the biggest diamond? I have a final word for you:
Sometimes when you choose a direction, it does not mean that it will be the future, but that it may be a different future.
Thank you!
Q&A
Q: I’m an Android developer and I know Flutter quite often. I have a question about code migration. Many applications have been developed for many years and the amount of code is quite large. Or any suggestions? It’s in a scenario where there’s a lot of code.
Answer: That’s a good question. The Flutter has two things. First, the new business is a better option, to start directly from scratch. Second, how to transform the existing business? It is recommended to try to choose relatively simple pages at the beginning, and only cut gradually. First, find relatively light pages to try to run through the road, and all the data indicators can meet the standards. Then try more, and gradually cut more into the page. It is not necessary to plan the whole project at such a big time. The risk is very high. It involves not only technical problems but also commercial problems.
Q: How do you share bitmaps between Native and engine?
Answer: Very simple, they are all in the same process, why can’t they communicate? Not physical isolation, but logical isolation. What is logical isolation? Writing code we’re at the engine level, and a lot of things in the engine can go directly to the address, and if you’re at the application level you might not be able to do that, but Flutter can be understood as a small system, like you can change the Android Framework and the underlying stuff, so what can’t you change?
Q: We have two platforms in one project. How do you solve the embedding problem and access problem of both platforms as a Model in the project, but the Flutter is referenced in the native?
Answer: You are actually asking about hybrid engineering. For the old business transformation must be inseparable from mixed engineering. For hybrid engineering, we also have a tool called FlutterW, which is a quick one-click access to hybrid engineering, such as containerization, which I mentioned earlier, is a quick access to hybrid engineering.
Question: I have an interesting question. I saw on the powerpoint that you used the Web. What I am curious about is that this thing is in the Build stage. As I said earlier, this thing is more likely to be an application on the mobile end, because it is more likely to do the same experience as the mobile end. So I want to know what your plan is for using the FlutterWseb? For example, is it a H5 downgrade scheme?
Answer: That’s a very good question. First of all, Flutter for Web is in a preview state, and bytedance, even in a preview, will follow. This is a technical follow-up. In addition, there are some internal projects in mainland China. Landing Flutter for Web is a technical attempt. You can also use Flutter for Web as a Fallback solution, which is a good alternative. With the development of Flutter, it is believed that next year for Web will be even better. The performance of the Flutter for Web will be worse with H5, although there are some optimizations, but it is still worse than H5. If you really use a large online application, I suggest you observe for a period of time, if your team is more manpower, you can also now access to do the corresponding transformation.
Live video
V.qq.com/x/page/q302…
More wonderful share
Shanghai salon review | core of bytes to beat on the Spark SQL optimization practice
Shanghai salon review | bytes to beat level how to optimize all nodes HDFS platform
Shanghai salon review | Apache Kylin principle is introduced to share with the new architecture (Kylin On Parquet)
Review | Shanghai salon Redis high-speed cached in the application of the large data scenarios
Bytedance Technology salon
Bytedance Technology Salon is a technical exchange event initiated by Bytedance Technology Institute and co-hosted by Bytedance Technology Institute and Nuggets Technology Community.
Bytedance Technology Salon invites technical experts from Bytedance and Internet companies in the industry to share hot technology topics and front-line practical experience, covering architecture, big data, front end, test, operation and maintenance, algorithm, system and other technical fields.
Bytedance Technology Salon aims to provide an open and free exchange and learning platform for talents in the field of technology to help them learn and grow and continue to advance.
Welcome to “Bytedance Technical Team”