preface

For e-commerce apps, pages developed using H5 technology account for a high proportion. Since the loading speed of H5 is very dependent on the network environment, it is very important to optimize the loading speed of H5 in order to improve user experience. Offline package is the most commonly used optimization technique. By downloading HTML/JS/CSS resources required for H5 rendering in advance, local cache resources are directly used during loading to avoid additional network requests and improve loading speed. This paper mainly introduces the team’s exploration in offline package technology solution and how the offline package implementation solution based on Prefetch reduces maintenance cost and development cost.

Existing programs

The off-line package technology has been quite mature until now. Offline package technology is mainly divided into two parts, one is the client offline package container, the other is the online offline package platform.

Offline package container

  • Resource request interceptionIntercept –H5Resource request, which is returned when there is a local cache resource
  • A resource cache– Resource download, resource cache policy, and incremental update policy

Offline package platform

  • Resource management– the configurationH5Page corresponding offline resources, public offline resources,CDNStores offline resource packages
  • Release system– Real-time publishing, grayscale capability, version control

Here are the common technical implementation methods:

Resource request interception mode

Android

Android is relatively uniform, mainly through the WebView own shouldInterceptRequestAPI intercept resource request, return the corresponding offline resource offline package function can be realized.

iOS

IOS is much more complicated to implement due to Apple’s limitations.

NSURLProtocol scheme

Use NSURLProtocol to intercept all requests made within webViews.

Problems existing in the scheme

The Body is missing

Because WKWebView itself uses multi-process mode, the WebView resource network request is not in the APP process. The current implementation of iOS system will lose the Body when intercepting HTTP network requests, so we need to deal with the problem of Body loss. One way is to replace the web API inside the WebView, such as Fetch/XMLHttpRequest, but it doesn’t cover all scenarios. Another way is to use the native API bridge for network requests, but this requires H5 adaptation and is somewhat intrusive.

Using private apis

WKWebView itself does not support network request interception. When we need to intercept a network request, we need to use the system private API to dynamically invoke the ObjC Runtime. There are certain audit risks, for example, the use of Apple audit is not allowed to be rejected. In addition, since the API is not exposed by the system, the internal implementation may change in the future.

WKURLSchemeHandler scheme

WKURLSchemeHandler is a new feature introduced in iOS11 that intercepts H5 network requests using this API.

Problems existing in the scheme

HTTP/HTTPS is not supported
  • HTTP/HTTPS is not supportedBecause –WKURLSchemeHandlerThe design of the API itself, which can only intercept custom protocols is not supportedHTTP/HTTPSThe agreement. One way is native loadingH5Use custom protocol orH5Internal resources use custom protocols. Another way to do it ishookSystem method supportHTTP/HTTPSAgreement, but this will bring certain risks and uncertainties.
Cookie problem

WKURLSchemeHandler does not handle set-cookies in the response, so it needs to handle them itself.

Body missing problem

This scheme also has the problem of Body loss.

The Local Server program

In Local Server mode, a Local Server is started when the APP is running. When requesting H5, the Local Server accesses the Local Server. The Local Server checks whether the Local offline resources are available.

Problems existing in the scheme

Virtual links
  • Virtual links– Because it needs to be usedVirtual linksAccess the local server, so it will bringCookies are synchronousSuch problems need to be solved
Resource consumption
  • Local servers have extramemory,CPUconsumption

PWA scheme

PWA provides a full set of Service Worker apis to implement offline H5 capabilities, including resource downloads, updates, caching policies, etc. However, the iOS system itself does not provide a default implementation, which requires self-implementation of a set of related Service Worker APIS, resulting in high complexity and workload.

Offline package management platform

Incremental update strategy

Offline package resources of an H5 page are usually aggregated into a ZIP package for download. To avoid full download caused by only partial resources being updated, differentiated update capabilities are required. Only changed resources need to be downloaded.

This section describes the PreFETCH solution

Design goals

After analyzing the common off-line package schemes in the industry, we have done a round of sorting for the design objectives of off-line package. Part of it is what the front end team wants, part of it is what we want to achieve:

Less invasive

  • H5 low invasion– No additional adaptation is required for accessing offline packagesNo sense. On the one handTo reduceFront-end adaptation costs and code complexity, on the other hand, help us drive more coverageH5Web page
  • Primary non-invasive– No need to use a specificWebViewThe container

Low maintenance cost

Because offline packages involve downloading resources in advance, you need to configure resource urls for downloading in advance. Existing solutions typically require a platform to manage these resources and configure a static resource file URL list for each H5 page that requires offline package capabilities. One problem is that each update requires manual maintenance of the entire static resource URL list, which we want to avoid as much as possible

Personal opinion: the better way here is to integrate the offline package system with the front-end publishing system, and automatically update the static resource list to the offline package resource management system when publishing.

Low runtime consumption

  • Low network consumption– Download only necessary resources to avoid unnecessary and duplicate resource downloads.
  • Low CPU/memory– Minimize memory and CPU consumption when not in useZero load

Low implementation complexity

  • Background management system– Temporarily unable to support the development of a complete offline package background management system due to human resources problems
  • Client container– The client implementation should be as simple as possible, so that the client can go online faster and avoid additional problems

The specific implementation

The idea is to use the prefetch capability of H5 browser. By aggregating offline package resources into a single HTML, the HTML is loaded in advance using a WebView after the APP starts, and the WebView downloads resources to the device. At the same time, it can directly reuse WebView’s offline cache ability and differentiated resource updating ability.

prefetch.html
<html>
  <head>
    <! -- Public Resources -->
    <link rel="prefetch" href="https://wq.360buyimg.com/js/common/dfd0ab35.js">
    <link rel="prefetch" href="https://wq.360buyimg.com/data/fontRegular.ttf">
    <! --A page resources -->
    <link rel="prefetch" href="https://wq.360buyimg.com/jxpp/app.css">
    <link rel="prefetch" href="https://wq.360buyimg.com/data/min.js">
  </head>
  <body></body>
</html>
Copy the code

H5 Offline package resource aggregation

As mentioned earlier, we don’t want H5 business development students to manually manage and maintain offline package resources, so we want to provide an ability to automatically aggregate resources. Reduce subsequent maintenance costs while minimizing resource downloads.

Check whether the offline package is enabled

Connect to the online H5 performance monitoring system, and automatically determine whether to enable offline package preloading according to access times and TOP rankings

Part of H5 can be added if it needs to be preheated

Aggregate resources
  • It is more accurate to calculate the resources to be downloaded based on the actual loading than manually maintaining the resources
  • By multipleH5Referenced resources are automatically identified as public resources

Tip: Generally, resource management is more difficult to manage, especially after long-term maintenance of public resources. In many cases, after adding resources, it is not known whether they are used or not.

Resource Aggregation Process

By running an automatic script, Puppeteer and Performance TimingAPI automatically calculates the offline package resources to be downloaded and updates them in time.

How to determine the first screen resources

Use the built-in browserPerformanceTimingAPI.domInteractiveIt’s the browser that does it for allHTMLIs parsed and the DOM build is complete. indomInteractiveThe previously loaded resource is bothblockingResources for the first screen rendering. At the same time, we need to filter out some resources that do not need to be cached. Currently, we only collect themJS/CSSBlocks rendered resources.

The client

The implementation of the client side is relatively simple. After the APP starts, a new WebView container is initialized and loaded silently in the background. Prefetch. Release the WebView container after loading, with no additional performance penalty thereafter. Although the download logic is retriggered with each startup, only differentiated downloads of resource files that do not exist in the local cache are performed.

Other optimization

Load the WebView in advance

Because initializing the WebView for the first time after APP startup will include initializing the Web engine, the initialization time will be higher. Therefore, we also initialized the WebView in advance when we pre-downloaded resources, and the initialization time could be reduced by 100-200ms when we opened H5 later.

Open the login mode in advance

Since most services H5 require login state, the APP needs to synchronize the native login state information to THE H5cookie when opening H5 for the first time, which will take an extra 302 jump time. We opened the login state in advance when preloading resources, and then opened H5 to reduce 100-200MS302 jump time.

Interface prepull

At the same time, it also provides the ability of interface pre-pull, which can pull the interface data on the first screen before loading to improve the loading speed.

Problems encountered during the implementation process

iOS

Do not support the prefetch

The iOS web kernel does not support Prefetch, so we use Preload for iOS instead. Link-prefetch is delivered on the Android platform and Link-preload is delivered on the iOS platform for differential processing.

Tip: Prefetch performs better than Preload. Prefetch has a lower download priority than Preload, avoiding impact on other network request speeds. Preload adds JS/CSS parsing to memory, resulting in additional consumption.

Preload does not support HTML

The iOS preload feature does not support the advance loading of HTML documents. This doesn’t really matter to us, though, because currently our BUSINESS H5 HTML usually does some server-side rendering logic and doesn’t support caching strategies. (for example, aggregate part of a common JS)

Multiple domain name resources are not shared

Other resources used by webViews in iOS for different domain names H5 cannot be shared. Such as https://www.jd.com/index.html and https://www.jingxi.com/index.html are the same website, inside have to use the same JS/CSS/images resources, but based on the iOS WebView caching policy implementation, Resources for each domain are managed independently of space and cannot be shared using repeated downloads. Because our own H5 support jd.com and jingxi.com dual domain name access, so we added the domain name replacement logic in the APP side, as far as possible to convergence our own business to jingxi.com domain name, improve the utilization rate of cache resources.

Tip: Even if you don’t use offline packages, this is a good optimization strategy.

The Android system

Insufficient disk space triggers a Crash

When prefetch is used to download resources, some devices Crash due to insufficient disk space. So we added an extra disk space check policy before downloading resources and did not download when the disk space was too low.

conclusion

Prefetch scheme

We do this by using the systemThe browserself-providedprefetchPreloading resource capability andHTTPOffline cache capability, to achieve a relatively lightweight offline package solution,H5The first screen performance improvement is basically the same as any other solution (except it’s not supported on iOS)HTMLOffline resources). Also through offline resourcesAutomatic statistics/Automatic updatesIn this way, no additional offline package resource management system is required, reducing subsequent maintenance costs.

Although the implementation cost and maintenance cost of this scheme are relatively low, there are still some shortcomings in the choice of implementation method, which need to be further improved. For example, it cannot intercept network requests to expand its capabilities. In addition, it is not controllable to rely on the browser’s own cache policy. For example, the Android browser has too many kernels and resources must comply with the HTTP cache policy. At the same time, the offline resource automatic statistics/automatic update capability is not easy to abstract a set of standardized solutions applicable to different companies. However, technical implementation solutions are often trade-offs and trade-offs, and this is a relatively low-cost implementation solution that we believe is currently available.

Value of offline packages

Personally, I don’t think the first-screen loading revenue brought by the offline package mode of downloading resources in advance is that high. The reasons are as follows: 1. Downloading too many offline resources in advance will cause more network consumption. 2. Most pages cannot be used offline (network access interface is required). 3. Offline packages are only optimized for the speed of the first load, since the resources themselves can be set to HTTP caching policies to avoid repeated downloads. The first screen loading of H5 page should pay more attention to the rendering performance of the page itself, such as JS/CSS parsing time, straight out or not straight out, the interface speed of the first screen, etc.

What is more valuable is how we can enhance more capabilities by intercepting network requests, such as providing HTTPDNS, native /H5 reuse image caching, etc.

Extended link

  • WKWebView request interception exploration and practice
  • Evaluate key render paths
  • How does offline Hybrid container open nearly 100% seconds?
  • Prefetch is supported
  • Preload is supported
  • WKWebView offline solution – implement Service Worker API