Flyfu Wang: Exploration of front-end offline

An airplane meme

One day, Xiao Ming gave a sudden feedback: “I found I couldn’t use your application last night… . What’s going on?” My friends and I were immediately shocked, thinking: “The old driver’s years of experience has a hunch, that is, you use posture is not correct… (300 words omitted) “. After a long investigation, the answer was:

Xiao Ming was on the plane last night.

In order to have fun on the plane in the future, the offline experience here needs to be optimized. Since then, a round-the-clock demand has been on the agenda.

The value of going offline

In an era of increasingly cheap data and constant references to cloud computing and 5G, some people feel that being offline is no longer necessary. When we talk about offline, we think of the old forest in the deep sea, the middle of nowhere. In fact, offline is very close to our lives and very frequent. Highways, subway tunnel, corridor, corner, and a variety of daily signal unstable region, these scenes have a large number of users through every day, every day there are millions users frequently because of network problems, the bottom of my heart poking fun at complained about our application, broken network offline is not our fault, but we can from the perspective of the user experience, Try to improve their anxiety when entering a weak or offline state, thereby bringing positive experience benefits to the product, improving user retention and word-of-mouth.

Optimistic UI

Speaking of improving User anxiety, it is important to introduce Optimistic User Interfaces. Optimistic UI is an interface response pattern that recommends that the front end update the UI before the server receives the response, and then change it to the actual result once the server returns.

For example, the user clicks the button, the front-end updates the data status as successful, the request arrives at the background, the server responds, updates the front-end data. Because 99% of responses are successful, only a small percentage of users need to fall back to the failed state.

Optimistic UI is not an advanced technology or something new, but an “offline first” mindset designed to improve the mood of the user experience.

Several common schemes of front-end offline

Application Cache

HTML5 was one of the first to provide a caching mechanism that allows Web applications to run offline. We use the Application Cache interface to set the resources that the browser should Cache, that is, to configure the manifest file. When the user is offline, click the refresh button, and the Application can also load and work normally.

This interface was quickly abandoned by the standard, partly because it was poorly designed, with slow updates, no fine-grained javascript controls, poor usability, and a lot of pitfalls if you didn’t follow its rules strictly. In its place is a more powerful service-worker.

service-worker

Because the Application Cache has been unable to effectively handle offline resource refinement, the service-worker interface was designed to provide a separate background JS thread compared to the Application Cache. Is a special worker context access environment. In the progressive Web application PWA, SW provides the most core support for the Network Independent feature.

With CacheStorage, we can populate cache resources as needed during the lifecycle of a SW installation activation, and then intercept HTTP requests and return cache resources or custom messages to the page in the FETCH event.

Service-workers implement true usability and security. First, it is invisible to the original Web application logic, which is similar to an intermediate interception service, and any errors in the middle fall back to the request-line logic. Second, it can only run over HTTPS to ensure security.

Sw had a fatal problem with our offline solution, which was ios WebView compatibility. Ios 11.3 + Safari supports WS, however, as apple has always done, UIWebView does not support service-worker by default.

Offline data

In fact, most of our offline scenarios will be in the local standalone app. With client capabilities, we can build the Web code package into the client ahead of time, and then use a code update mechanism to solve the front-end code caching problem. The offline code loading and updating logic itself is not complicated. The following is a simplified diagram. In specific business scenarios, there are also issues to be considered, such as whether gray users, code version and data synchronization.

One of the complexity of Offline solution lies in the processing of Offline data and how to make minimal modification to the old code that was not considered as “Offline First” at the beginning of design. Basic functions in Offline state are given priority and further enhanced online. Considering the decoupling of offline and online logic, we should consider the offline solution based on the principle of minimizing the intrusion of the original online logic. Let’s take a look at common offline data front-end solutions.

PouchDB

PouchDB is a cross-platform javascript database that encapsulates IndexDB and WebSQL-compliant front-end processing internally.

Generally speaking, the front-end pouchDB is used for offline processing, and the back-end CouchDB is used to facilitate two-way data synchronization.

The Sync interface is dedicated to before and after data that is out of Sync:

PouchDB is a good solution for off-line data processing in small and medium-sized projects, especially full-stack development where the back-end can be taken over by the front end. The problem with this scheme is that it is still more than 130 KB after compression, and depends on a specific background scheme, which is not universal.

Redux-Offline

For projects that use Redux data management, the quickest way is to use Redux-Offline. The idea is to use Redux Middleware to listen for every acton change. Then serialize the data that needs to be offline to the local store (the compatible order for Web browsing is IndexDB – webSQL – LocalStorage), and restore the data from the local store to the store first when the next page is refreshed. The advantage of this solution is that the API interface that needs to be cached can be quickly configured into the middleware, and the full combination of Redux features is very good for applications that want to easily display offline data first.

The problem with this approach is that it is not flexible enough to manipulate the data, and the data stored locally cannot be easily shared with other non-Redux logic. In the case of a large amount of offline data, simultaneous read and write operations and simultaneous serialization of a large amount of local data may also cause performance problems, and may not be appropriate for scenarios with frequent data changes.

Redux is combined with IndexDB

To achieve fine-grained control of the data without too much intrusion into the original online logic, we can replace the backend data with IndexDB for data storage and reuse the original Redux for front-end data processing.

Local storage of business data needs to be paid attention to is reasonably abstract the data used by the business, and then according to the basic principles of database design local table, here can also talk with the background students, to avoid the omission of design problems.

Due to the rough IndexDB native operation API, we assembled a set of generic DB low-level operation libraries and abstracted the API interfaces to be shared by all businesses in the form of git sub-repositories. Here, first of all, simplifies the front-end business logic layer DB local literacy, sorting, and so on, is advantageous for the related project sharing, secondly to DB of abstracting is also in order to better facilitate the business itself can not rely on IndexDB itself, can be combined with the client features, to replace the underlying database and optimization provides a convenient, or for pure web side, For downward compatibility can use WebSql, LocalStorage and other compatible provides an extension.

For the front-end code architecture, how to use Redux to quickly and elegantly convert the original online request background interface to local read and write?

It is more appropriate to extract a separate layer of Redux middleware and pass in the API that needs to be offline during initialization in the form of configuration files. Then, read and write the DB in the middleware and assemble the data to the next reducer, which can be called Offline middleware. In order to further reasonably decompose API parameters, we also need to abstract another middleware, API Middleware, from the interface request layer before offline middleware, so that the API parameters of offline middleware have been decomposed and can be directly used as query DB. It can also serve background requests.

Data synchronization is preferred when offline

We have been able to make the interface that needs to be offline offline through offline middleware through configuration, so there are two data update methods. The first one is to wait for the background data to return through normal logic when online, asynchronously synchronize to the local database, and then render. Read from local when judged offline. There is also an optimistic UI mindset that configures an offline interface to first manipulate data locally, render the UI, and then synchronize server-side data with local data. Obviously, the latter offline first approach is more sensible.

Data synchronization is classified into local to background synchronization and background to local synchronization. Incremental change logic needs to be added to handle offline user data changes due to other reasons, such as data movement or deletion when a user logs in to multiple devices. (Front-end incremental offline change involves many details and service-related considerations, which are not detailed here.)

How do you log local data changes and then synchronize them to the background? Here we need to define an abstraction for data changes, such as Change

The main function is to define change types, fields and so on. Each time the data arrives at offline middleware, the changes are registered through a data synchronization manager and de-synchronized at the appropriate time. The data synchronization manager receives change, performs diff management, determines whether data has changed, and performs deduplication management. Finally, asynchronous synchronization tasks are triggered. Synchronization may fail, so timeouts, retries, and fail-backs need to be paid attention to to ensure the synchronization is transactional.

Safe storage

Storage security includes data encryption security and storage size. For symmetric encryption, front-end view, the client must know the key, the key itself can not open the encryption problem, in theory, with the server communication offline state, any data can be viewed in the front-end offline, no matter what encryption means, data can be restored. Pure front-end data encryption has no reliability, but access rights can rely on the Same origin policy of the IndexDB browser for secure data isolation. Avoiding plaintext storage and making it more difficult to restore data directly is the direction of thinking.

One easily overlooked security issue is iframe, which has access to the IndexedDB library of the source it is embedded in, so we need to ensure that all resources on the page are trusted.

The storage scheme using IndexDB involves a storage size issue. The maximum storage space of the browser is dynamic, with a total of 50% of the available disk space and 20% of the available space per site. Writes beyond this limit will result in data being deleted and resulting in serious data loss. Since IndexDB storage space is not available directly from the browser itself (string calculations are unreliable and highly inaccurate), limiting the number of stores was one of the ideas from a product statistical perspective, and a better solution would be to use an end-to-end store such as LarvelDB to prevent data loss. For the pure Web end, it is also worth trying to expand the form of browser plug-in (such as Google Doc) to ensure data security more reasonably.

“

Going offline is a feature that many front-end projects are not designed for because it is not cost-effective for most purely presentational Web projects. But as a tool-based, creative application, offline can be a feature that has long-term benefits. Imagine an artist, looking at the scenery on the plane, suddenly having an Epiphany, opening our product to create, indicating that it cannot be used, but a big loss….

AlloyTeam welcomes excellent friends to join. Resume submission: [email protected] For details, please click Tencent AlloyTeam to recruit Web front-end engineers.