Years ago, I talked with many friends about the value of end-to-end intelligence, both touted and disparaged, and each of them had their own reasons. Here, as an interested party, I would like to share my views on end-to-end intelligence and try to answer the questions of whether, why and how it can be used. The article does not contain technical dry goods, aims to provide communication consensus for products, algorithms, engineering, please read it as appropriate.

The views expressed in this article are personal and do not reflect the views of Bytedance.

Who murdered the A.I.

During the Spring Festival, the famous victim Newton Dundun’s favorite, half Buddha immortal, updated a ke (Qi ǎ) pu (Fan) video [Half Buddha] who murdered artificial intelligence and robots. Excerpts from the script are as follows:

Artificial intelligence is now touted as omnipotent. But there are two core difficulties: one is data sources and data standards; One is the data collection cycle for application scenarios. Artificial intelligence relies on algorithms, which require vast amounts of data to train. The problem is that everyone is bragging about how good their algorithms are, but in fact algorithms are not the weakness of AI. And now open source algorithm model a lot of, a lot of so-called algorithm engineers, just model tuner plus SQL Boy. The days of high-paying algorithms are over, and companies are slashing headcount for these jobs. The biggest problem with ai landing in industry has never been algorithms, but rather the industry's serious lack of data accumulation, and the current data standardization is very poor, and the data in many fields is directly blank. When you understand the landing mode of AI, you can realize why ai has been touted a lot so far, but the best place to use it is Internet companies' big data killing, loan sharks, information cocoon room and let the delivery man to repair fairy. Because these industries are data-oriented and standardized, data sources directly violate user privacy. Define the business by setting goals in terms of how much money you make and how long you engage, and then feed the algorithm data. This logic makes a lot of money on the Internet, because everything is in the cloud.Copy the code

Although Teacher Banfo is not a big name in the field of artificial intelligence, his views on artificial intelligence in the video still represent the views of many practitioners. In short, the video argues that AI relies on algorithms, and algorithms rely on standardized data and continuous data collection, and that the current bottleneck in the industry is not algorithms, but data. During the Spring Festival, my communication with my partners in the traditional medical and financial systems also confirmed this point of view.

The joys and sorrows of human beings are not the same, and the problems they face are also different.

Although for many traditional industries such as medical treatment and manufacturing, the lack of digitalization is a huge obstacle to their application of ARTIFICIAL intelligence, but the experts and scholars struggling in the front line do not stop because of difficulties ah — the original accumulation of data is not complete, how can we get rid of the fate of being stuck?

So ai has never been murdered. Instead, it has been on the move.

, on the other hand, for many Internet companies was born in the digital age, genetic data driven is engraved on his bones, if can’t independent construction good data system, it should at least know use ali cloud, tencent cloud, the growth of the volcanic engine provides analysis and visualization monitoring and other infrastructure, otherwise, is absolutely difficult to survive in the fierce competition.

Therefore, may not be able to fully cover partial, the prospect of end-to-end intelligence, but also need a line of us bow into the game, to give their own answers.

In addition, “big data killing, usury, information cocoon house and let the delivery man to repair fairy” is not the original sin that artificial intelligence should bear. Artificial intelligence is a universal tool, and how to design and optimize the target is still up to human beings.

End-to-end intelligence at the intersection

End-to-end intelligence can be literally broken down into “end-to-end” and “intelligent.” The term “end to side” is mainly used to distinguish between service ends. It can refer to mobile terminals alone or IoT, but this article focuses on mobile terminals.

Many point to Apple’s release of the iPhone 4 in 2010 as the start of the mobile development boom. According to the statistics of QuestMobile, the monthly year-on-year growth rate of mobile traffic is 4.9% in 2018, 2.3% in 2019 and 1.7% in 2020, slowing down year by year, and the demographic dividend is gradually fading. Although the major companies are still hot in the traffic inventory space, but we are already in the “nO one wants iOS” industry voice resume hard to find.

Self-driving cars, brain-computer interfaces, and the chip embargo have all made headlines over the past year, but the common theme behind them is artificial intelligence. At the age of 70, artificial intelligence is a late bloomer. As a result, the number of resumes has skyrocketed year after year, and the employment situation is moving. Rao is so, zhihu on the highly praised career answer or choose to adhere to.

What new species could be assembled by shivering mobile phones in the corner and artificial intelligence in the spotlight?

And more importantly, will the future be good?

Data, computation, and connectivity

The Internet is all about connectivity, and AI is all about data and computing. Here, from the perspective of information processing, we review the evolution of data, connectivity, and computing over time.

In ancient times, the data possessed by human beings were very limited both in quantity and dimension, and information could only be exchanged through migration or clan wars. In this period, the patriarch of a clan can often coordinate the whole clan, and the lack of data and connectivity is the main development bottleneck. After that, the long period of land annexation, population accumulation, ethnic integration and industrial and commercial development resulted in a surge of affairs to be dealt with, while communication tools such as galloping roads, canals, post stations and beacon towers could not keep up with the speed of territory expansion. In this case, even though the emperor and the general are gifted, they can no longer plan the affairs of large and small by themselves. Therefore, there were three public officials and nine ministers, three provinces and six ministries to share power and responsibility. Prefectures, counties, provinces, roads, prefectures and other administrative divisions were controlled by local officials nearby. At this point, data is no longer the main bottleneck, and the lack of single-point computing power is to some extent hidden by layering and division of labor, while connectivity, especially the ability to organize during special periods of war, still determines the rise and fall of countries to a large extent.

Until modern times, with the emergence of the telegraph and the Internet, the shackles of connection were really broken, and the flow of global information became possible. Since the advent of MS-DOS in 1985, the computer industry has raced for 30 years, aided by Moore’s Law and Andy Beale’s law, providing the computing power needed for more data and wider connections. With the development of computers, digitalization has spread throughout the whole society. New business models such as social networks and e-commerce have emerged, and the concept of data as king has been deeply rooted in people’s minds along with the development of big data. Then came the mobile and smart waves. With GoogLeNet’s success in ImageNet 2014, visual, auditory, verbal and other information has been incorporated into the map of data that can be processed by computers, and computers have begun to have practical computing power to understand the real world. Mobile devices provide enhancements across the board — deep lenses, microphones and sensors provide extra dimensions of data, WiFi, 4G and 5G make connectivity ubiquitous, and ever-changing chips provide necessary computing power.

Did you find anything? The development of data, connection and calculation is not linear. Different development curves combine problems and bottlenecks in different periods. End-to-end intelligence is a solution to break through the bottleneck.

Similar logic might apply to Edge, but the problems faced by three-level and two-level connections would certainly be different.

Besides, the challenge

So what are the challenges facing AI in today’s mobile Internet?

Personal opinion. The challenges facing ai today are different from those that follow big data and use it to make business decisions in virtual digital worlds, and those that are emerging to penetrate dimensional walls by perceiving the real world. Let’s use the concepts of computational and perceptual intelligence to distinguish between the two.

Let’s start with computational intelligence. In addition to the new generation of intelligent Internet finance, most Internet companies still need to consolidate the business foundation before optimizing efficiency through intelligence. Therefore, most Internet companies need to go through a long or short period of data system construction. Domestic first-line factories have basically crossed this step, but there are strong and weak points in data processing capacity – most companies are still exploring the model design, the head of the company has squeezed the value of the model to the limit, a few thousand a year to improve is a good result. At this time, how to legally explore the undeveloped data resources, how to improve the data return time within the limited cost increase, how to optimize the data system, improve the efficiency of the algorithm, it has become the problem that the head companies need to think about.

And then perceptual intelligence. It is true that images, videos and voice messages are becoming more sensitive to data collection. Companies are using open datasets, crowdsourcing platforms and livestreaming to get data. In addition to data cleaning, sample annotation and model verification, the threshold is not low, but it is not without solutions. The expansion of multimedia scenarios also means higher traffic and computing overhead, which puts forward higher requirements for connection and computing at the same time, and the burden of connection and computing even begins to become unbearable for some businesses. But scenario definition may be the hardest of all — the value of AI depends not directly on the model, but on the extent to which it can improve operational efficiency and reduce labor costs within the scenario. The definition of “useful” for ai models varies from scenario to scenario. For example, for pneumonia identification, whether to assist doctors in batch or replace expert verification is a completely different requirement. For new businesses, there is already plenty of proven low-hanging fruit to be picked; Cutting-edge enterprises, on the other hand, need to find new scenarios where ROI is feasible and risk is manageable under the constraints of data, connectivity, and computing.

What’s 1 plus 1?

It’s so important that it should be repeated for three times.

End-to-end intelligence and cloud intelligence are not in conflict.

End-to-end intelligence and cloud intelligence are not in conflict.

End-to-end intelligence and cloud intelligence are not in conflict.

Compared with the cloud, endside is simply a dreg with only five combat effectiveness, and its storage and computing power are not even of the same magnitude. It is just like the ancient county officials without much power, but close to the thing, it has the possibility to change the design. So think of end-to-end intelligence as the new weapon in your Arsenal, and imagine yourself as the architect of institutions, tweaking the way data and computation are distributed to smooth out the gaps in data, connectivity, and computing capabilities through elaborate layering.

“What? I wanted to use behavioral timing in the model, but the guys in the background said it was really hard?”

Compared with the cloud, the end-to-end data storage is limited by space, but the data reading and writing of the end-to-end data are almost real-time, while the data in the cloud must be reported, cleaned, shunted, processed, and dropped, with a delay of minutes at the earliest. At the same time, end-to-end data is naturally recorded in time sequence, which is incomparable to cloud. It is possible to make clear the importance, storage period and real-time performance of data, and reasonably allocate the data storage and processing between the clouds.

As an additional note, data can flow, and not just from cloud to cloud, but from cloud to cloud, as long as data privacy and data security are not at risk. For example, if the content is returned in a list result, the rest of the content can be adjusted on the side when negative feedback such as “not interested” is generated.

“What? Record a video with smart subtitles and every sentence is delayed by as much as half a second?

Compared with the cloud, the computing capacity of the end side is much weaker, so it needs to balance user tasks and computing tasks, and be careful to prevent overheating and stall. But in this age of intolerant progress bars, where delays or delays mean lost users most of the time, it makes sense to do calculations on the data production side where the network connection is out of control. In order to lower the equipment cost, more extensive user coverage, many inference engines in the performance “holy grail war” infinite volume, many algorithm teams in the quantization, pruning, NAS model optimization road secretly competing.

When the network is shelved, the connection does not disappear, but shifts to an end-to-end internal loop from data to computation. The bridge between data and computation is just not a network anymore. Computation can be triggered by timing, by a frame of video, by a click, or even by the result of another calculation. The sky is wide open, and you can take care of the whole end-to-end business link. Every magic number, every dead rule and the original cloud intelligence can be the starting point of your transformation.

“What? 50,000 live broadcast of the festival, the server even content security will be unbearable?”

There are many workarounds for computing allocation, and although the power of a single client is limited, the power of all the clients combined is far greater than that of the server. If the model can be migrated directly to the end side, the natural cost is lowest; If the model cannot be migrated directly, it is possible to split parts of the model as long as the benefits outweigh the costs of splitting. If this is not possible, you can also set up a filtering model on the client side, using additional small models to avoid requests that do not meet expectations; Really not, on the end of the data processing and then submit, save a little calculation. At the same time, do not neglect the use of intermediate results, if the results of disassembly model, filtering model or data processing can give feedback to users, it can also improve user experience to a certain extent.

Furthermore, in addition to reasoning, if end-to-end sample construction can be achieved, training can also be migrated to end-to-end. End-to-end training has a wider data selection space, but the sparser samples also bring higher challenges to learning rate control. However, once the training hurdle is crossed, it means to realize the thousand thousand modules that are difficult to be realized in the cloud due to storage and training costs, and then the federated Learning, or Meta Learning, is no longer so far away.

Efficiency and Experience

The next problem is how to find bottlenecks in data, connectivity, and processing capabilities of the business, and properly deploy the intelligence of the end cloud.

“Meeting user needs” is too general a term to guide practical decisions. One of my favorites is huang Hai of Phonak Capital, who said, “There are too many companies in China that aim to improve efficiency and too few that create experiences.” Both directions — efficiency and experience — apply equally to end-to-end intelligence.

Efficiency and experience are not a single point but a global one, so we must not find the “hammer”, see where there are “nails”. As Sun Tzu said, “Know yourself and know your enemy, and you can fight a hundred battles with no danger of defeat.” To improve efficiency and experience, first of all, it is necessary to clarify the status quo of business links — what is the user’s usage behavior, what is the main target of the behavior, what data is used, what control branches are used, what data or rules are used in the control, and what data are generated accordingly. Then, based on the obtained information, evaluate the impact of existing and potential control points on efficiency and experience and the cost of transformation, and specify the transformation plan and target. Finally, when it comes to the actual execution, you have to

Always pay attention to details and fill in the gaps.

Since it is a global project, cooperation is usually unavoidable. If you cannot complete the evaluation independently, you may as well ask upstream and downstream students for help. Generally speaking, engineering students at the end and cloud can confirm the validity of data at both ends, product students can judge the value and priority, and algorithm students can evaluate the feasibility and effect of the algorithm.

Let me give you an example. At a time when everyone thought it was just beauty effects, something changed. On the anchor side, the audio part has intelligent noise reduction, the video part has ROI area recognition and beauty effect, and the coding part has bandwidth prediction and intelligent selection of bit rate resolution. On the client side, it also predicts the user bandwidth, dynamically adjusts the playback bit rate, and further improves the quality of the image. Such a combination of data intelligence and perceptual intelligence, on the one hand, enables users to watch live broadcast without getting stuck, and on the other hand, saves the cost of enterprises on the network. This is how the competitive difference between intelligent and unintelligent begins.

Discussion & Recruitment

Is there a future for end-to-end intelligence? This is actually not a good question. The end-to-end intelligence is just a tool to solve the problem. Only with the definition of the scenario problem can we move from artificial intelligence to business intelligence. Can end-to-end intelligence help business intelligence break the shackles of data, connectivity and computing that this age has placed on it? Of course can!

So much for Spring Festival. Engineering and practice is a huge project, we did not open source, let me steal lazy bar.

Welcome to discuss. Welcome to byte Client AI interested students private letter consultation, together with us to conquer the business intelligence of the stars of the sea!

Mobile Team in Hangzhou recruitment link is as follows:

• Flutter frame direction Leader/ Engineer
• Flutter Engineer – Client Infrastructure Department
• Cross-platform Development Engineer
• Engine Development Engineer
• DevOps Senior Development Engineer – Mobile Infrastructure
• Mobile build specialist
• Terminal Intelligent Application Engineer (iOS/Android)
• Inference engine engineer
• Terminal intelligence algorithm engineer
• Server R&D engineer – Terminal Intelligence