It was first posted on my blog github.com/mcuking/blo…
background
In the hybrid development mode of H5 + Native, the problem that makes people complain most is probably the blank screen during loading the H5 page. The following diagram depicts the entire process from WebView initialization to the final rendering of the H5 page.
The current mainstream optimization methods mainly include:
-
WebView initialization: this process takes approximately 70 to 700ms. When the client starts up, a global WebView can be initialized in advance and hidden. When the user visits the WebView, the WebView is directly used to load the corresponding webpage and display it.
-
Sending interface request to the back end: When the client initializes the WebView, the network requests data directly from Native. When the page initialization is completed, the data requested by its agent is obtained from Native.
-
For loading JS dynamic stitching HTML (single page application) : can use multi-page packaging, server-side rendering, and pre-rendering at the time of construction.
-
For the size of page resources: Lazy loading and other methods can be adopted to separate the parts that need larger resources, and then asynchronously request the separated resources after the whole page rendering is completed, so as to improve the overall page loading speed.
Of course, there are many other aspects of optimization that I won’t go into here. This article focuses on the process of establishing a connection to a static resource server and then receiving a front-end static resource. Because this process is so dependent on the user’s current network environment, it is also the least controllable factor. When the user is on a weak network, the page load speed may reach 4 to 5 seconds or longer, seriously affecting the user experience. Offline package scheme is a mature solution to this problem.
Technical solution
First of all, elaborate the general idea:
We could start with the page needs to be packaged and preloaded static resources to the client installation package, when users to install, then extract resources to local storage, when the WebView to load a H5 page, intercept all HTTP request, check to see if the requested resource in the local existence, if there is a direct return resources.
The following is the overall technical plan. I use Jenkins for CI/CD by default. Of course, other ways can also be adopted.
The front part
Related codes:
Offline package packaging plug-in: github.com/mcuking/off…
Front-end project of application plug-in: github.com/mcuking/mob…
The idea is that the WebPack plug-in will emit hooks (before generating resources and exporting them to the directory), and the Compilation object (which represents a single build and build of resources) will traverse and read the resources generated by the WebPack package. The information for each resource (traversal scoped by file type) is then recorded in JSON for a resource map as follows:
Json example of resource mapping
{ "packageId": "mwbp", "version": 1, "items": [ { "packageId": "mwbp", "version": 1, "remoteUrl": "Http://122.51.132.117/js/app.67073d65.js", "path" : "js/app. 67073 d65. Js", "mimeType" : "application/javascript"},... }Copy the codeRemoteUrl is the address of the resource on the static resource server, and path is the local relative path of the client (by intercepting the server request corresponding to the resource, hitting the relevant resource locally based on the relative path and then returning).
Finally, the JSON file mapped to the resource and the static resources to be localized are packaged into a ZIP package for use by the subsequent process.
Offline package management platform
Related codes:
Offline package management platform Front and back ends: github.com/mcuking/off…
File differential tools: github.com/Exoway/bsdi…
From the above description of offline packages, it is not difficult to see that there is a missing question, that is, how to update the offline package resources in the client after the static resources update? Do you want to send a new installation package? Wouldn’t that take away the dynamic nature of H5?
The offline package platform is designed to solve this problem. Here I take the mobile-Web-best-Practice front-end project as an example to explain the whole process:
The offline package corresponding to the mobile-web-best-Practice project is named Main. The first version can be preset into the client installation package as mentioned above, and uploaded to the offline package management platform, which saves offline package files and related information. A JSON file named packageIndex is also generated, which is a collection of information about all the relevant offline packages, and is mainly provided for the client to download. The general content is as follows:
{
"data": [
{
"module_name": "main",
"version": 2,
"status": 1,
"origin_file_path": "/download/main/07eb239072934103ca64a9692fb20f83",
"origin_file_md5": "ec624b2395a479020d02262eee36efe4",
"patch_file_path": "/download/main/b4b8e0616e75c0cc6f34efde20fb6f36",
"patch_file_md5": "6863cdacc8ed9550e8011d2b6fffdaba"
}
],
"errorCode": 0
}
Copy the codeData is a collection of information about all offline packages, including the version and status of offline packages, as well as the URL address and MD5 value of files.
When the mobile-web-best-practice update is made, a new offline package will be packaged via offline-package-webpack-plugin. At this point, we can upload the offline package to the management platform. At this point, the version of the offline package main in packageIndex will be updated to 2.
When the client starts and requests the latest packageIndex file, if the version of the offline package main is larger than that of the local offline package, the client downloads the latest version from the offline package platform and uses it as the resource pool for querying the local static resource file.
The reader may also have a question at this point, that is, if the front end is only one change, the client still needs to download the entire new package, does not waste traffic and prolong the file download time?
To solve this problem, we can use a file difference tool – bsdiff-nodejs. This node tool calls the BSDIff algorithm implemented in C language (calculate diff/patch package based on binary file comparison). When the offline package of version 2 is uploaded to the management platform, the platform will perform diff with the previously saved offline package of version 1 to calculate the differential subcontracting of 1 to 2. The client only needs to download the differential package, and then use the tool based on BSDIff algorithm to generate the offline package of version 2 by patch with the offline package of local version 1.
The general principle of offline package management platform is finished, but there are still areas to be improved, such as:
-
Adding the Log Function
-
Added the statistics function of the offline packet reach rate
.
The client
Related projects:
Integrated offline package library android project: github.com/mcuking/mob…
The offline package library of the client is only developed on the Android platform at present. The library is a secondary development based on WebPackageKit (android offline package library developed personally). It mainly realizes a multi-version file resource manager, which can support the preset of multiple front-end offline packages to the client. The source code for intercepting the request is as follows:
public class OfflineWebViewClient extends WebViewClient {
@TargetApi(Build.VERSION_CODES.LOLLIPOP)
@Override
public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request) {
final String url = request.getUrl().toString();
WebResourceResponse resourceResponse = getWebResourceResponse(url);
if (resourceResponse == null) {
return super.shouldInterceptRequest(view, request);
}
return resourceResponse;
}
/** * hits from local and returns resources *@paramUrl Resource address */
private WebResourceResponse getWebResourceResponse(String url) {
try {
WebResourceResponse resourceResponse = PackageManager.getInstance().getResource(url);
return resourceResponse;
} catch (Exception e) {
e.printStackTrace();
}
return null; }}Copy the codeIntercepts HTTP requests by replicating the shouldInterceptRequest method of the WebviewClient class and looking locally for the corresponding front-end static resource, and returning it directly if so.
Partial problem solving
1. Can offline packages be automatically updated?
The current resource is automatically packaged by CI machine and then deployed to the static resource server, so how to upload to the offline package platform? I have considered using the interface to automatically upload the front-end resource to the offline package platform when it is packaged. However, it was later found that the feasibility was not high, because our front-end resources need to be updated by manually modifying the Docker version through operation and maintenance after the test stage. In the case of automatic upload, the offline package platform has uploaded an unverified front-end resource, but the static resource server has not updated. Therefore, you still need to manually upload the offline package. Of course, readers can choose the appropriate way to upload according to the actual situation.
2. How to distinguish which App the offline package belongs to in the case of multiple apps?
The appName field is added when the uploaded offline package is filled in. When a JSON file of offline package list is requested and the appName field is added to query, the offline package platform returns only the list of offline packages belonging to the App.
3. Do I have to download the offline package when the App starts?
Of course, you can do more, such as when the client is connected to Wi-Fi, or switch from background to foreground for more than 10 minutes. This setting item can be configured in the offline package platform and can be made globally valid or customized for different offline packages.
4. If the client offline package has not been downloaded and the static resource server has deployed the latest version, will the client display the old version? If this change is an interface request change, that will not cause the interface error?
Don’t worry about that. If the HTTP request doesn’t hit any front-end resources, the code above lets it go to the remote server. So even if the local offline package resources are not updated in time, you can still ensure that the static resources of the page are up to date. That is to say, there is a bottom-of-the-pocket solution, if something goes wrong, it goes back to the original loading mode of the requesting server.
5. If the offline package version of the client is 1, and the latest version of the corresponding offline package of the offline package platform is 4, that is, if the version difference is greater than 1, is patch consolidation performed locally by downloading differential subcontracting?
Currently, the offline package platform developed by the author only differentiates adjacent versions. Therefore, if the local offline package version and the latest offline package platform version are not adjacent, the full package of the latest version will be downloaded. Of course, you can make the difference between the uploaded offline package and the previous 3 or more versions according to your needs, so that the client can choose to download the corresponding version of the difference package, such as download 1->3 difference package.
6. If the offline package is not only offline JS, CSS and other resources, but also offline HTML, will there be any problems?
As an example, suppose the client requests the online and offline package version at app startup and every two hours. Shortly after the app just requested the online offline package version, the online front page resources were updated, and the online offline package was also updated. When the user accesses the page, the client does not know that the online resource has been updated, so it still intercepts the HTML resource request and looks it up from the local offline package. Because there is no hash in the HTML file name, the file name remains the same even if the page updates, so the corresponding HTML file can still be retrieved from the local offline package and returned, even though the HTML file is older than the online file. Resources such as JS/CSS referenced in old HTML are also old resources, resulting in the user’s view of the page remains old. You can’t update to the latest page until the app is restarted or the client requests the online offline package version again nearly two hours later.
The main source of this problem is that clients do not know when online resources will be updated and can only do timed polling. If the server proactively notifies the client, for example, by pushing, to request the latest version of the offline package when the online and offline package is updated, the timely update can be ensured. (It may take some time to download the offline package.)
At this point, readers can think about a question: is it really important that the front end of the page is updated in time? This involves the choice between the user’s page opening experience and the timely update of the page, which can be compared to the native APP. Generally, the native APP will download the update only after the user agrees to update, and the version used by many users may not be the latest. Therefore, the author believes that as long as we can do a good job of back-end interface compatibility, so as not to appear that the page does not update, the online interface parameters of the request are changed or even abolished, leading to the page error, the page cannot be updated in time or can be tolerated.
Besides, the average user doesn’t use the app for very long, and the next time they open the app, the client will download the latest offline package. My company also has this problem, but it does not affect the actual use of users. Therefore, it is recommended to take the HTML file offline to completely speed up the page opening.
7. WkWebview on iOS has no API to support direct interception of web page requests. How to implement the offline package scheme?
The author inquired the cloud music offline package on the tip of the iOS, is through private API – registerSchemeForCustomProtocol registered HTTP (s) scheme, which can access to all of the HTTP request (s), For more information, see the following article:
Nanhuacoder. Top / 2019/04/11 /…
Since WKWebView performs Network requests independently of the main NSURLProtocol Process, the NSURLProtocol Process cannot intercept requests from web pages in webView. (Note: UIWebView request, NSURLProtocol can intercept)
If the registration by registerSchemeForCustomProtocol HTTP (s) scheme, Then all HTTP (S) requests initiated by WKWebView will be processed via IPC from Network Process to main Process NSURLProtocol, and all Network requests can be intercepted.
However, MessageQueue is used for communication between processes. The Network Process will encode the request into a Message, and then send it to the main Process NSURLProtocol through IPC (inter-process communication). For performance reasons, HTTPBody and HTTPBodyStream are discarded by encode.
One solution mentioned in the article is as follows:
One problem is that the HTTP header itself is limited in size, causing scenarios such as uploading images to fail. Here is a way to go:
When initializing the wkWebview, inject and execute a section of JS whose main logic is to duplicate the open and send methods mounted on the globally mounted XMLHttpRequest prototype.
Generate a string identifier based on the timestamp in the open method, mount it to the XMLHttpRequest instance object, add it to the second parameter Url, and then execute the original open method.
As for the send method, it takes the body of the HTTP request and the identifier attribute of the open method that is mounted to the instance object, combines it into an object, and calls the native method to store it in the client’s storage.
When an XHR request is intercepted in the main process NSURLProtocol, the identifier is retrieved from the requested Url, and then the previously saved body is retrieved from the client’s store based on the identifier. This solves the missing body problem.
Of course, if the project uses the fetch method provided by the browser natively, remember to copy the fetch method as well.
At the end
So far, the general principle of the whole scheme has been described. For more details, readers can refer to the project link provided in the article. All the codes have been hosted on my Github.
This is the fulfillment of a long-cherished dream of mine: to implement an offline package solution that is completely open source. Finally, I hope to be helpful to you