An overview of the

In the second half of 2019, in order to migrate the Didi Chuxing of WeChat Wallet/Alipay Jiugongge entrance to a small program, the team carried out a lot of function upgrades and repairs on the small program. In the whole process, a series of problems and challenges have been encountered and overcome, among which the problem of packet volume is particularly prominent. Next, we will comprehensively introduce the efforts and precipitation of Didi Chuxing applet in volume control.

background

WeChat requires that the total volume of the small program package shall not exceed 12M, and the volume of the main package and a single sub-package shall not exceed 2M. Alipay’s calculation method for small package volume is slightly different from WeChat’s, but the whole is much the same.

From 2018 to the beginning of 2009, the services carried by Didi Chuxing mini-program were only online car-hailing, and the business needs were less, which could be handled within the main package. In the second half of the year, in order to migrate WeChat wallet/Alipay Jiongge entrance to small program, small program began to add many business lines, such as bus/agent drive/car service/bicycle/ride, etc. At the same time, the business demand of online car hailing also needed to make up comprehensively, and the business volume and code volume exploded together.

Didi Chuxing covers a wide variety of travel services, including express car/private car/taxi/luxury car/ride-sharing/bicycle/agent drive/hitch ride/bus/car life and many other business lines. The most important and frequently used pages of the whole Didi Chuxing appletlet are the home page and the order detail page. The home page carries the demand expression of each business line, while the order detail page of each business line carries the specific travel order display logic. In addition, there are various functional pages such as Personal Center, Marketing Page, Settings, History Schedule.

According to the product logic of Didi Chuxing, the demand expression logic of all business lines is carried on the home page. In order to provide a good switching experience, the top guide of a single page is adopted to display business lines on the home page. That is, each line of business provides a demand expression component in the home page, and when the user switches to the top of the line of business, the corresponding line of business component is switched out.

In this design, the demand expression logic of all business lines is concentrated on the single page of the home page. As a result, in the process of business iteration, the volume of the main package carrying the home page grows rapidly, which soon hits the upper limit of 2M for a single package of the small program platform, bringing huge obstacles to the subsequent business iteration and development. Therefore, the control of packet volume is a big problem we are faced with in the process of small program development.

The volume control

In the following, we will introduce a series of optimization and control practices for the volume of the small program package during the iterative process of the development of Didi Chuxing small program.

Basic optimization means

For small programs, the basic packet size optimization means include: resource compression/code redundancy removal/resource CDN/asynchronous loading

In Web development, Webpack provides a large number of code optimization capabilities, including dependency analysis, module de-duplication, code compression, Tree Shaking, Side Effects, etc. These capabilities can easily complete the work of resource compression and code redundancy removal. Small drops travel program based on drops open source application framework Mpx (https://github.com/didi/mpx) for development and compilation of Mpx framework built entirely based on webpack, compatible webpack internal ecology, natural can use the ability to optimize the package volume.

Some static resources (such as images and videos, etc.) are supported to be loaded by CDN address in the applet. We will compress the relevant resources to the CDN as much as possible to avoid the occupying of the packet volume by these resources.

In the small program scenario, asynchronous loading cannot be carried out easily through script label as in the Web, but the small program platform supports the subcontract loading scheme in the later stage to achieve this ability. Since subcontract loading is a unique technical specification of small programs, Webpack cannot directly support it. Therefore, the MPX framework specifically provides good adaptation support for this technical specification, and we will elaborate on the application of this capability in the following article.

In addition, the MPX framework also carried out many adaptions of package volume optimization for small program scenarios, such as minimizing the package volume occupied by the framework runtime (56KB after compression), packaging and building the referenced pages/components on demand, declaring common styles for style reuse, extracting common modules in subcontracting, and so on.

With these capabilities of the MPX framework, it is possible to build a preliminarily optimized small package with little additional configuration.

WeChat Developer Tools has a similar option to check “Compression Obscure when Uploading Code”, but in Developer Tools the volume calculated when uploading code is the volume of the current project code directly, not based on the compressed volume. Therefore, if you are developing with native applets, your source code will most likely be further compressed to save space.

Analysis of the volume

Although the framework has provided a lot of optimizations in terms of volume control, as the business iterated we found that the main package volume was still too large.

After the main package volume is too large, we need to figure out what is in the main package? Why are they so big?

Those of you who are developing with native applets or other non-Webpack-based frameworks may have to look at the file size on your hard drive. As a result, the size ratio of each module may not be intuitive. A Webpack plug-in such as Webpack-Bundle-Analyzer can be used to assist with the analysis.

For example, if this is a demo written using the Mpx framework, you can see this interface by using NPM run build –report:

You can see that the demo project consists of third party libraries such as Moment/Lodash/Socket-Weapp/Cor-JS. The size and interdependence of the individual libraries can also be clearly seen.

For Didi Chuxing applet, you can also see a similar picture, and you can see what code the whole project is made up of.

In addition, the front-end development of Didi Chuxing has always adopted the “source code compilation”, so that the common dependencies in the whole project can be realized only one and shared together. In short, it also helps to reduce project code size. Related: https://github.com/DDFE/DDFE-blog/issues/23

To perfect play the effect of the source code to compile, need to build together to establish a complete set of source of upstream and downstream of the ecology, such as additional qualifications to rely on party in public announcement rely on, you should use peerDep or devDep to declare some public reliance on, the mutual dependence should be unified statement in the main project, avoid the version put on two different public dependence, That would actually increase the volume. As Didi Chuxing appletlets involve many business lines and teams, some teams may not know this, so the above deterioration scenario may actually occur in the code. By following the diagram, it is easy to spot such problems and push the team to clear up these duplicated dependencies.

At the same time, according to the volume analysis diagram, we focused on the analysis of the larger files, carried out a round of business code combing and simplification, deleted some useless codes, simplified the WebSocket message body description files and so on.

Configuration of the subcontract

Subcontracting is a scheme similar to Web asynchronous introduction given by small programs. Some pages not needed at the initial entry can be put into the subcontracting, and the subcontracting can be downloaded only when they jump to the corresponding page. Putting these pages and their attached resources into the subcontracting can effectively reduce the volume of the main package.

The MPX framework provided preliminary support for the subcontracting specification in the early stage. The resource access rules were consistent with WeChat and should be output to the main package or subcontract mainly according to the directory of the resource stored. After having this ability, we took out the travel page to the subcontractor, and about 200 K of space was extracted.

With the successful split of the itinerary page, we began to subcontract all the non-front page code, such as start and end selection and personal center. In addition, the access of some lines of business is through NPM. We also try our best to put all non-home code of these lines of business into subcontract.

Here is a side note. Thanks to MPX’s early design of Packages, it is very convenient for businesses to develop independently and integrate easily. WeChat packages can be configured in a json configuration similar to packages, so the user can add a query to the packages to mark the name of the package.

After dismantling each subcontract, the whole project structure is roughly as follows:

After the completion of the initial subcontracting work, a total of nearly 400K of space was removed from the main package to the subcontractor.

Subcontract resources fine management

As mentioned above, the early subcontracting rule of the MPX framework is to collect resources under the subcontracting path into the subcontracting completely in the manner of WeChat. The NPM managed resources, which are in the node_modules directory and do not belong to any subcontracting path, are all collected into the main package.

For example, before we had travel page subcontracting, the self-owned state management store of travel page was all under the path of travel page subcontracting, and would be collected in the travel page subcontracting. The itinerary page also uses the packaged didi-socket library. This library is a public NPM package, and since its path is under node_modules, it will be collected in the main package, even if it is only used in the itinerary page package.

In some early designs, the resources of itinerary page and the home page are separated and stored independently in their respective paths. The major part of the first phase of subcontracting processing is also the itinerary page, which just fits the characteristics of MPX’s early subcontracting processing, so it can be collected in the itinerary page subcontracting better.

As the business iterates, a large number of subsequent lines of business are accessed through NPM, and there will be a large number of NPM package resources, all of which are in the node_modules directory, so they will all be collected into the main package.

So the MPX framework underwent a number of modifications:

  1. During the dependency collection process for a build, we mark the collected dependencies and note which subcontracts it was introduced by. Once it has only one subpackage in it, it will be exported to that subpackage.
  2. We will automatically generate cacheGroups of each subcontract in SplitChunksPlugin according to the user-defined subcontract configuration, and extract the reusable modules of the subcontract into the bundle under the subcontract.
  3. For components and static resources, if they are referenced by more than one subpackage and not referenced in the main package, to ensure optimal volume of the main package, multiple copies of these resources are exported to the corresponding subpackage without occupying the main package volume.

This way, no matter where the resource referenced in the subpackage was originally located, when it is finally exported, it will be exported to the subdirectory of dist as much as possible to avoid taking up space in the main package.

After this change, the project structure looks the same as before, but thanks to MPX’s improved ability to handle subcontracted resources, we were able to successfully export the NPM resources referenced in the line of business subcontract to the subcontract directory where they were located.

Cover scheme

Later, Didi Chuxing applet needed to replace the original WebApp entrance in WeChat/Alipay. The business lines accessed by the applet increased rapidly, and the package volume increased rapidly.

The main reason for the increase in volume of this section, as mentioned earlier, is that all lines of business are connected to the home page for display. This is also determined by business characteristics. Didi Chuxing provides a variety of travel product lines, including express car, private car, taxi, luxury car, ride-sharing, bicycle, agent drive, hitch ride and other products, which users may need to choose repeatedly. This process also preservers information such as start and end vehicle types, and requires an in-page switching component plus a very complex set of large state management to run smoothly. Unlike some e-commerce/information platforms, which split different functions into different pages and let users enter the sub-pages through the menu of the home page for operation, the home page only carries the entrance and has less business logic, so it will be much easier to subcontract.

Therefore, each line of business should provide home component for access. This component will be used on the home page, so it will not be removed from the subpackage under any circumstances. Finally, the entire volume of the main package section of the home page can be divided into two parts: the base library and the business code. The volume ratio of the two is about 1M for the common dependency base library and 1M for the business code.

Such a large base volume is mainly due to the numerous business lines and business teams of Didi Chuxing, and each party has its own base to rely on. For example, the long link communication PB data description file dependent on online ride-hailing, the large number calculation library based on map, the runtime of CML framework dependent on ride-hailing, the communication gateway library dependent on ride-hailing, and the common component library and polyfill, etc.

Therefore, the problems faced by Didi Chuxing appletlets at that time could not be quickly solved by pure technical solutions in a short period of time. Therefore, we made an engineering architecture adjustment, which can be called the cover page solution, to solve the main package problem.

Simply put, the cover plan is to make a cover with the Didi Chuxing Logo as the launch page, and once the page is loaded, it immediately jumps to another page, which really carries the business, and it is placed in the subcontract.

What this means is that all that is left in the main package is the infrastructure/libraries that all parties depend on, and the business is pulled out to the subcontractor.

This is the structure diagram of the project after the completion of the cover scheme. A large chunk of the business logic of the front page has been extracted into the front page subcontract.

As a result of this shifting operation, we can have a main package space of 2M to multiply the basic public library, and a subcontractor of about 2M to multiply the “big home page” of Didi Chuxing, which integrates various businesses mentioned above. At that time, there were almost 1.2m main contractors and 800K+ main subcontractors.

The most outstanding point of this transformation is that the volume growth generated by subsequent business iterations is almost all in the main business subcontracting, and the remaining 1.1m + space for business iterations is still relatively abundant. The volume of the main package can remain unchanged for a long time under ideal conditions, so the volume of the main package will not exceed the standard repeatedly due to the continuous development of business requirements, and the volume of the main package will not need to feel anxious.

Of course, as you can see, the scheme itself does not subtract any volume, just a change of position. In addition, this cover page scheme actually has some drawbacks, such as the first screen business display will be slow, because of the load of more content, but the small program itself has a better ability to cache resources, so it is acceptable.

Instead of blocking demand iteration and product line access due to volume problems, this solution can at least solve the problem of whether or not. Our development team will continue to follow up and pay attention to the volume issue in the future, and see if there will be any product solution changes or some solutions provided by the applet itself to further optimize this part.

conclusion

The MPX framework does a lot of work on packet volume control and has very good support for small program subcontracting in the NPM scenario.

On the basis of framework support, Didi Chuxing applet team controlled the huge and complex volume of Didi Chuxing applet package within the limited range of the platform without impeding business development by sorting out business dependencies, making full use of subcontracting, adjusting interaction schemes and a series of other means.

Hope this article can bring some inspiration to the small program developers who have problems with the package size, welcome to leave a comment.