preface

Web offline technology, as its name implies, packages H5/CSS/JS and resource files and delivers them to App in advance, so that App actually loads local files when loading web pages, reduces network requests to improve webpage rendering speed and achieve dynamic update effect.

In view of the current situation, there are many offline package schemes. This paper will list the four most common offline schemes in the market, discuss and analyze them, and select the best scheme to build the offline package function. If you have the need to optimize the rendering speed of H5, you can refer to it. This paper only does the technical selection and scheme principle analysis, and the following chapters will select the optimal scheme for in-depth discussion and concrete implementation. The table of contents is a subsequent extension.

plan

  1. Load directly by obtaining the sandbox H5 path
  2. Request interception based on NSURLProtocol
  3. Custom scheme registration interception based on WKURLSchemeHandler
  4. Load local resources from the local server

The selection

Scheme 1: Load directly by obtaining sandbox H5 path

Load h5 directly, which is what the famous Cordova framework is based on.

  • 1. Put all h5 files in one folder.

  • 2. Dump this folder into the project code in a relative path.

  • 3. Obtain the local file path.

The solution is to unpack the front-end code deployed on the server directly into the local sandbox. When loading JS, directly load the HTML in the local sandbox for offline loading. Each front-end module is defined as an application, marked with an ID and sent to the client. When users click the corresponding module, they will search for the corresponding offline resource in the sandbox according to the ID and load it in seconds.

  • Pros: Simple.
  • Disadvantages:

    1. In fact, as you can see from the screenshot, when accessing the local HTML we can see the actual path isfile:///... /index.html. This is in useThe file agreementAccess TO HTML, some HTML styles do not support file protocol, there will be missing in style and function, there will be some API differences, the front-end development of the code may be downloaded to the sandbox, resulting in some resources can not be used, some adaptation problems.
    1. Accessing local resources can also lead to resource path leakage and security issues.
    1. There are also some browser security Settings that don’t work.
    1. The inability to implement cross-domain resource requests prevents front-end developers from accessing external CDNS.

File protocol & HTTP protocol: The file protocol is mainly used to access files in the local computer, like opening files through the resource manager, local, that is, file protocol is to access file resources on your local computer. HTTP access to local HTML is a local HTTP server, and then you access the local server on your computer, the HTTP server to access your local file resources.

Browsers sometimes handle the two protocols differently. For example, in some web pages, the file protocol is called to open an image. This function is blocked by the browser’s security Settings, because by default, HTML is a hypertext language running on the client, and the server cannot perform local operations on the client. Even local operations such as cookies require security level Settings. If you need to load resources from an external CDN, such as livereload, browserSync, etc., loading external files from the local file system will fail due to the browser’s same-origin policy and will throw a security exception.

In general, this scheme will cause serious intrusion to the front end, restricting the front end to load js, CSS, image and other resources through the relative path. Moreover, the cross-domain problem of file protocol leads to the failure to introduce external CDN, which will limit the development of the front end. Although it is easiest to use, it is not a good scheme.

Scheme 2: Request interception based on NSURLProtocol

Since loading local resource files directly is not the best solution, can we consider another solution based on NSURLProtocol interception? Of course it works, but read on:

On UIWebView, protocol interception is indeed our preferred solution. Create a subclass and implement the proxy method of protocol in the subclass to achieve interception of all requests, including HTML for CSS, JS, IMG and other resource loading requests.

- (void)startLoading
{
    
    NSData *data = [NSData dataWithContentsOfFile:filePath];
    if (mimeType == nil) {
        mimeType = @"text/plain";
    }

    NSHTTPURLResponse* response = [[NSHTTPURLResponse alloc] initWithURL:[[self request] URL] statusCode:200 HTTPVersion:HTTP / 1.1 "@" headerFields:@{@"Content-Type" : mimeType}];

    [[self client] URLProtocol:self didReceiveResponse:response cacheStoragePolicy:NSURLCacheStorageNotAllowed];
    if(data ! =nil) {[[self client] URLProtocol:self didLoadData:data];
    }
    [[self client] URLProtocolDidFinishLoading:self];
}
Copy the code

This perfectly solves the h5 resource request problem.

WKWebView performs network requests in a separate process from the app process. The request data does not pass through the main process. Therefore, using NSURLProtocol directly on WKWebView cannot intercept requests. Of course a private API can solve the problem:

/ / only iOS8.4 above available Class CLS = NSClassFromString (@ "WKBrowsingContextController"); SEL sel = NSSelectorFromString(@"registerSchemeForCustomProtocol:"); if ([(id)cls respondsToSelector:sel]) { #pragma clang diagnostic push #pragma clang diagnostic ignored "-warc-performSelector -leaks" // Register HTTP (s) scheme, NSURLProtocol [(id) CLS performSelector:sel withObject:@" HTTP "]; [(id)cls performSelector:sel withObject:@"https"]; #pragma clang diagnostic pop } }Copy the code

However, there is still a defect, the BODY data of the POST request is cleared. Because WKWebView performs network requests in a separate process. Once the HTTP (S) Scheme is registered, Network requests are sent from Network Process to App Process so that NSURLProtocol can intercept Network requests. In the design of Webkit2, MessageQueue is used to communicate between processes. The Network Process will encode the request into a Message, and then send it to the App Process through IPC. The HTTPBody and HTTPBodyStream fields are discarded for performance reasons.

If you use Get request to intercept offline resources, there is no problem. After intercepting the request, NSHTTPURLResponse* Response is generated for local resources, just like the scheme above. But there’s another risk of using a private API: rejection.

As an aside, as far as I know, Baidu App Android is using the request blocking method, but it is Android, see the following picture:

Image source: Baidu App-Android H5 First Screen Optimization Practice

The figure above shows the analysis of steps 11 and 12. When WebView parses HTML, it can find and intercept resource requests, return corresponding cached resources and render them. In fact, this scheme does not work on iOS, Android can use their own browser, you can modify the browser, such as Alipay UC, Baidu T7 and so on. There’s no magic browser in iOS apps, unfortunately, which means we can only use what Apple’s dad opens up.

In summary, this solution does not invade the front end and the front end can still be developed step by step without any changes. However, there are still risks in body interception and use of private API, but as far as I know, this scheme is also being used by some projects, so IT is recommended.

Scheme 3: Customize scheme registration interception based on WKURLSchemeHandler

WKURLSchemeHandler WKURLSchemeHandler is an iOS11 solution for handling custom requests, but it does not handle regular schemes such as Http and Https.

Opens WKWebViewConfiguration setURLSchemeHandler: forURLScheme: function, you need to specify a custom scheme and a used to handle WKURLSchemeHandler callback custom object.

According to the comments, an exception will be raised if you register an invalid scheme or use a scheme already handled by WebKit, such as HTTP, HTTPS, file, etc. It is best to use WKWebView’s handlesURLScheme: class method to check the availability of a given scheme, so as not to introduce some unknown problems. It’s easy to use:

if (@available(iOS 11.0, *)) {
        BOOL allowed = [WKWebView handlesURLScheme:@ ""];
        if (allowed) {
            WKWebViewConfiguration *configuration = [WKWebViewConfiguration new];
            // Set URLSchemeHandler to handle requests from a specific URLScheme. CustomURLSchemeHandler needs to implement the WKURLSchemeHandler protocol to intercept requests from customScheme.
            [configuration setURLSchemeHandler:[CustomURLSchemeHandler new] forURLScheme: @"customScheme"];
            WKWebView *webView = [[WKWebView alloc] initWithFrame:self.view.bounds configuration:configuration];
            self.view = webView;
            [webView loadRequest:[NSURLRequest requestWithURL:[NSURL URLWithString:@"customScheme://"]]]; }}else {
        // Fallback on earlier versions
    }
Copy the code

WKURLSchemeHandler provides two callback functions handled by the CustomURLSchemeHandler object above:

- (void)webView:(WKWebView *)webView startURLSchemeTask:(id <WKURLSchemeTask>)urlSchemeTask;
- (void)webView:(WKWebView *)webView stopURLSchemeTask:(id <WKURLSchemeTask>)urlSchemeTask;
Copy the code

The request object of urlSchemeTask can get the corresponding URL of the request. If it is our customized scheme, we can intercept it, map it to the corresponding local resource through the URL, and load the local resource.

If the local resource does not exist, then build a request object directly from the URL to access the server. If the local resource does exist, then load the local resource directly and use it as in the second scenario:

- (void)webView:(WKWebView *)webView startURLSchemeTask:(id<WKURLSchemeTask>)urlSchemeTask {
    NSString *urlString = urlSchemeTask.request.URL.absoluteString;
    // Locate the local resource and map it to the local resource address filePath
    
    NSData *data = [NSData dataWithContentsOfFile:filePath];
    NSHTTPURLResponse* response = [[NSHTTPURLResponse alloc] initWithURL:urlSchemeTask.request.URL statusCode:200 HTTPVersion:HTTP / 1.1 "@" headerFields:@{@"Content-Type" : @"text/plain"}];
    [urlSchemeTask didReceiveResponse:response];
    [urlSchemeTask didReceiveData:data];
    [urlSchemeTask didFinish];
}
Copy the code

In fact, this solution solves the problem of resource interception very well, and can do the same as the second solution. It doesn’t seem to be a problem. But it still has drawbacks:

    1. Because the custom scheme used is not HTTP, it still does not solve the cross-domain problem.
    1. Since scheme is customized, for the front end, scheme needs to be set as our customScheme, which will bring a lot of transformation to the front end, so it still incurs into the front end.
    1. As mentioned above, Android does not need such a costly detour as iOS, so Android may not need this custom scheme, which will lead to the serious problem of differentiation with Android.
    1. Due to API limitations, only iOS11 and later can be supported.

So WKURLSchemeHandler’s interception scheme isn’t very friendly either.

Solution 4: Load local resources from the local server

According to alipay’s article “Alipay Mobile Terminal Dynamic Scheme Practice”, offline package is described as follows:

When the H5 container makes a resource request, it uses the same URL to access local or online resources. H5 intercepts the request first. After intercepting the request, the following happens:

1. The H5 container uses local resources if there are resources available locally to satisfy the request.

2. If there is no local resource that can satisfy the request, the H5 container uses online resources. Therefore, a WebView is imperceptive whether the resource is local or online.

It can be seen that Alipay does not adopt the above three schemes, because none of the above schemes can make WebView unaware except protocol interception. As far as I know, Alipay should adopt the local server scheme at present. The difference between THE HTTP protocol and the local file protocol has been described in detail in the first solution. If you can use THE HTTP protocol to load local resources, this can make the front-end as much as possible “insensitive” to the offline package, that is to say, the front-end does not need to modify scheme. There is no need to consider whether there will be some problems caused by the file protocol, and the framework implementation differences caused by the platform differences of the interception API can be ignored. In this way, a copy of the front-end developed code can be deployed on the server and uploaded to our offline package platform. That’s why it’s called “insensibility.”

  • Advantages: advantages are said before, with the network server loading style and function is completely consistent, do not invade the front end, the front end does not care about the current page is offline or not offline, do the maximum no perception. Of course, there are pros and cons, and this is not a perfect solution.

  • Disadvantages:

      1. Additional local servers need to be set up, and the path of HTML files need to be processed.
      1. There are cost issues for local server setup, local server management issues, such as server on, off time and so on.
      1. It is also unknown to me whether local servers will bring other problems. Not all teams can build their own servers to deal with them like Alipay.

The implementation of this scheme can refer to the processing of “Realizing WKWebView Offline Resource Loading Based on LocalWebServer”, but several problems are also mentioned at the end of the article:

  • Resource access permission security is incorrect.
  • Service restart takes time during APP switching.
  • Problems with power and CPU usage during service operation.
  • Multithreading and disk IO problems.

These questions are also unknown to me. If you have a mature solution to set up a local server, please leave a message.

This article aims to analyze an optimal solution to build the core functions of the offline package, but because some friends raised some optimization problems such as preloading, so I have selected several optimization solutions from ‘Bang’s’ blog for reference.

Fallback technology

Off-topic: From the above mentioned alipay article, there is a paragraph we can analyze:

Fallback technology comes into being to solve the problem of unavailable offline packages. When each offline package is released, an online version is synchronized to the CDN. The directory structure is the same as the offline package structure. The FALLback address will be delivered to the local device along with the offline package information. If the offline package is not downloaded, the client intercepts the page request and redirects the corresponding CDN address to switch between online and offline pages at any time.

The unavailability scenario should be that the offline package is unavailable, not updated, the resource is corrupted, the MD5 does not match, or the validation does not pass, etc.

    1. If the local offline package does not exist or is not up to date, the system blocks and waits for the latest offline package to be downloaded. This solution has the worst user experience because of the relatively large volume of offline packages.
    1. If there is an old package, the user directly uses the old package this time. If there is no resynchronization blocking wait, the update is not timely and the user cannot use the latest version. (As far as I know, wechat applet is this scheme)
    1. You can create an online version for the offline package. The files in the offline package have a one-to-one access address on the server. If there is no offline package locally, you can directly access the corresponding online address.

The third solution should be Alipay’s Fallback technology, which can solve the above problems. Of course, the first two options are not inadvisable, or depends on the requirements and scenarios.

Common Resource pack

Each package will use the same JS framework and CSS global style. It is too wasteful to repeat these resources in each offline package. You can create a common resource pack to provide these global files.

Preload webview

Whether iOS or Android, local Webview initialization takes a lot of time, can be pre-initialized Webview. There are two types of preloading:

First preload: Initializing a Webview for the first time in a process is not the same as initializing it for the second time, which is much slower. The reason is expected to be that after the initial initialization of the Webview, even though the Webview has been released, some global service or resource objects shared by multiple WebViews are still not released, so the second initialization does not need to regenerate these objects to make them faster. We can pre-initialize a Webview at APP startup and release it so that when the user actually goes to the H5 module to load the Webview it will be faster.

Webview pool: You can reuse two or more WebViews instead of creating a new Webview every time you open H5. However, this way to solve the problem of clearing the previous page when the page jumps, in addition, if there is a MEMORY leak in JS on an H5 page, it will affect other pages and cannot be released during the running of the APP.

Preloaded data

Ideally, when the offline package is first opened, all HTML/JS/CSS will be cached locally without waiting for network requests, but the user data on the page still needs to be pulled in real time. We can make an optimization here, and request the data in parallel while the Webview initialization, which takes some time. There are no network requests at this time, so parallel requests at this time can save a lot of time.

The client initiates a request when the Webview is initialized, the request is managed by a manager, and the result is cached when the request is completed. Then the Webview requests the URL that has just been preloaded after the initialization, and the client intercepts the request. Go to the request manager mentioned earlier and return the content directly if the preload is complete, or wait if it is not.

Use the client interface

Network and storage interface if the use of WebKit ajax and localStorage will have many limitations, it is difficult to optimize, you can provide these interfaces to JS in the client, the client can do on the network request like DNS pre-resolution /IP direct connection/long connection/parallel request and other more detailed optimization, Storage also uses client interfaces to optimize read/write concurrency/user isolation. In the early web pages rendered by the server, JS was only responsible for interaction, and all contents were directly in HTML. To modern H5 pages, many contents already rely on JS logic to decide what to render, such as waiting for JSON data requested by JS, and then stitching it into HTML to generate DOM rendering on the page. So the page rendering display has to wait for the whole process, there is a time here, reduce the time here is also within the scope of white screen optimization. Optimization can be done by artificially reducing THE JS rendering logic, or more radically, going back to the original, where everything is determined by the HTML returned by the server without waiting for the JS logic, called server rendering. Whether or not to do this optimization depends on the business situation, since it can lead to changes in development patterns/increased traffic/increased server-side overhead. Some of the pages in Hand Q are rendered using a server-side method called dynamic straight out.

conclusion

As for these four schemes, there are advantages and disadvantages. As for the selection, I prefer NSURLProtocol interception and local server. Of course, still want to refer to their own needs, with respect to application, it is ok. There’s still a lot of work to be done to make a good Hybird framework, from the Alipay project to the Nebula project, to learn more about the dynamics of Q and Nebula. I don’t know if you have noticed that not only hand 100, including headlines, Tencent news, has been rendered before the page is not all pushed out, indicating that there is a pre-loading of H5 page processing, which is also worth our in-depth discussion. This, of course, depends on specific needs and manpower. As for offline package handling, these are all the solutions I can think of so far, and there is a summary of their pros and cons. If you have any suggestions or better solutions, please leave a comment.

Open source address: WKJavaScriptBridge (offline package introduced later)