First, the soul torture

Before we get started, some soul searching:

  • What process does front-end code go through from TSX/JSX to deployment and access by users?

  • What are the considerations, metrics, and optimization points in the above process to meet complex business requirements?

  • Most of you probably know about strong caching/negotiated caching, but what caching strategies should be used for various front-end artifacts (HTML, JS, CSS, IMAGES, etc.)? And why?

    • If a negotiated cache is used, but static resources are not updated frequently, how can you avoid wasted requests during the negotiation process?
    • How can static resources be updated if strong caching is used?

  • How should the supporting, front-end static resources be organized?

  • How does the accompanying, automated build & deployment process integrate with the CDN?

  • How do I prevent the front end from going online and affecting users who have not refreshed the page?

  • How to do a second rollback of a newly released version that has an obstructive bug instead of waiting 20 minutes or more to deploy again?

  • How to achieve a pre-delivery environment, except front-end resources are all online environment, variable control in the front-end environment?

  • How to facilitate the deployment of AB tests?

  • How do you implement one set of front-end code and release it into multiple sets of environment artifacts?

  • How to release products according to features for users to use, gradually expand the gray scale of features, and minimize the influence (that is, there are multiple feature products online simultaneously)?

  • CDN domain name suddenly hangs, how to achieve second level CDN downgrade repair instead of redeploy all services once again?

With these issues in mind, this article will try to explore a systematic front-end deployment solution in 2021.

PS: The article is reprinted from < Byte Architecture Front End >juejin.cn/post/701771…

Static resource organization

A simple page

Let’s start with a simple static page. As we all know, front-end resources consist of HTML, JavaScript and CSS. Suppose we have a simple page, using Nginx as a Web server, the resource organization structure is roughly as follows:

In this case, you only need to upload static resources such as HTML, JavaScript, and CSS to a directory on a Web server (such as Nginx) through FTP software, and start Nginx to do simple configuration for users to access it.

As soon as the user visits, state 200, the page renders, and the front end is pretty simple, right?

Using the cache

But on closer inspection, the user requests foo. CSS, bar. CSS, etc., static files every time they are accessed, even if the files are unchanged. It is a waste of bandwidth and also affects the performance of the first screen of the page. So in the early days of the Internet, when the network bandwidth was tight, computer pioneers developed a variety of caching strategies over HTTP.

Browser Caching: Brower Caching enables the browser to cache previously requested files so that they can be reused for next access, saving bandwidth, improving access speed, and reducing server pressure.

Negotiate the cache

One strategy is for the browser to ask if the server has changed and use the old resource if it hasn’t. After all, the communication cost of “ask” is far less than the cost of each resource reload. The general process is as follows:

Negotiation cache: sends a Request to the server. The server will judge whether the negotiation cache is matched according to some parameters of the Request Header. If yes, the server will return the 304 status code with a new Response Header to inform the browser to read resources from the cache.

At this point, after using the negotiated cache, the Network looks like this:

Note: Negotiation caching can generally be implemented on the server side by setting ResponseHeader such as last-modifed and ETag. Note: 304 status code indicates that the resource has not been changed. You can use the browser cache.

Strong cache

In this way, through negotiation cache, we have greatly optimized the network request when the resource has not changed, saving a lot of bandwidth, the first screen performance of the website has also been well improved, meizi! However, a closer look reveals that there is still a negotiation process, with a hundred negotiation requests for a hundred static files. When the resources are not changed, we who pursue perfection should also optimize this negotiation request, after all, there is no harm without buying and selling! The strong cache corresponds to the negotiated cache, and the process is as follows:

Strong cache: The browser does not send any requests to the server, reads the file directly from the local cache and returns Status Code: 200 OK.

At this point, the general conversation process of strong cache is shown as follows:

Note that while the cache is in effect, the browser is talking to itself, independent of the server.

At this point, after setting the strong cache, the Network looks like this:

From DiskCache: Reads data From a disk. From MemoryCache: The fastest way to read data From memory. Note: Strong caching is generally implemented on the server by setting cache-control :max-age, Expires and other responseheaders.

With the use of strong caching, negotiated requests are eliminated, and the performance of site loading is at its peak. A fun!

Appendix: Negotiation cache and strong cache details

Strong/negotiated cache details

Note: the implementation and use of strong cache/negotiated cache should be understood first. Master it later.

Cache update problem

Since pages (index.html) are frequently updated, static resources are relatively stable. Therefore, one caching strategy we can deduce is that index.html is suitable for negotiation caching, and relatively stable & infrequently updated static resources (JS, CSS, IMAGES) should eliminate negotiation requests and use strong caching. However, the problem soon becomes that we don’t even let the browser make requests, but the cache hasn’t expired yet. What if we find a bug and want to update foo.css? Do you want to set the cache to take as long as possible, but also want to be able to update at any time? Want a horse to run and not feed him?

I believe you will soon come up with an idea to add a version number to the resource! Such as throughqueryAdd version number, every time online unified change version number is done. At this timeHTMLAs shown below:

Note that there is only one file foo.css in the server at this point.

The advantage of using the same version number is that it is simple and fast, but the disadvantage is that if we only want to update foo.css, the bar.css cache is also invalid, resulting in wasted bandwidth. You should be able to quickly figure out how to bind the contents of a file to the version number (URL) and change the version number (URL) when the contents of the file change, so that precise cache control can be achieved for each file. What’s relevant to the contents of the file? Message summarization algorithm, the summary information of the file, the summary information and file content one by one correspondence, has a cache control basis can be accurate to the granularity of a single file. Now, let’s change the URL to one with file summary information:

We could call this way this wayquery-hashWhen the subsequent release goes online, only the files to be changedURLWill be updated to achieve precise cache control, perfect!

Note that there is only one file foo.css in the server at this point.

Issues caused by overwritten publishing

However, if we were to go online as described above, it would soon become Fatal, and every time we update it, there would be a disaster. As a review, the site has only one static file, which is deployed in a directory on the Nginx server, and uses query-hash to implement precise cache control per file. What is the problem? As a refresher, when we changed the foo. CSS style, the FOO. CSS URL in the HTML was updated to the latest hash, and the foo. CSS & index.html file stored on the server was overwritten to the latest (version V2), It looks like both HTML and static resources are updated, but there are no extremes. That is:

  1. Deploy static resources first. During the deployment, V1 HTML will access the new static resources of V2 and cache them using v1-hash.
  2. The HTML is deployed first. During the deployment, the old static resources of V1 are accessed by the V2 HTML and cached according to the V2-hash.

As shown in the figure below, different versions of HTML and different versions of static resources match each other to generate exception cases.

Green: Path for normal access and cache establishment. In red, static resources (V2) are deployed and V1-HTML accesses V2 static resources and caches Case. In black, HTML (V2) is deployed and v2-HTML accesses V1 resources and caches Case

For question 1, there are two cases:

  1. The user has a local cache. In this case, the access is normal.
  2. If there is no local cache, static resources of V2 version are loaded and cached based on hash of V1 version. The user reported an error. When the HTML version V2 is deployed, it is restored when the user accesses it again.

For problem 2, there will be a serious Case:

V2 HTML. V1 static resources are cached based on V2 Hash. The page will fail and continue to fail until the cache expires. Until the user manually clears the cache, or the cache expires, or a future V3 release updates the static resource version. Otherwise the user will keep making errors.

The problem with the above solution stems from the fact that the static resource has only one copy each time it is publishedOverlay publishing, resulting in page and static resources matching error! The problem solving solution is also extremely simple to useNon-coverage publishing, a simple modification method is to summarize the document (hash) toURLIn the,query-hashInstead ofname-hash.

The HTML now looks like this:

In this way, each deployment of the full deployment of static resources, and then gray deployment page, can be more perfect to solve the cache problem.

At this point, multiple files foo.[$hash].css will exist on the server

Combined with the CDN

We were happy to have the site up and running, but we were still deploying static resources under the Nginx server directory, and then a new problem arose. As time went on, non-overwrite deployment caused files to grow and hard drives to become strained. Moreover, the files are stored in a directory on the Nginx Web server, which deeply coupling Nginx, websites and deployment processes together. CDN technology cannot be used.

CDN is a content distribution network deployed at the application layer. It uses intelligent distribution technology to distribute to multiple nodes according to the location of users and the principle of nearby access to achieve multi-point load balancing. To put it simply, users can access nearby with faster access speed. Large companies do not need to build a storage server with super bandwidth, but just use multiple CDN nodes with normal bandwidth. The common realization of CDN is that there is a source server, and multiple CDN nodes are synchronized from the source.

How to combine CDN with Nginx and other Web servers? The answer is to deploy static resources to the CDN and forward traffic from Nginx to the CDN, a technique we call “reverse proxy”. At this point, the traffic direction of user access & the process of R&D construction and deployment is roughly as follows:

At this point, our overall deployment plan needs to be further transformed in three steps.

  1. Build time based on environment variables, willHTMLStatic resource address plusCDNThe domain name.
  2. Upload static resources when the build is completeCDN 。
  3. configurationNginxTo forward static resource traffic toCDN.

Among them, article 1 and 2 are related to the adjustment of the construction process. Taking Webpack as an example, we need to make the following configuration modification:

A. Set 'output' to 'content-hash' & 'publicPath' b. Configure 'webpack-html-plugin' to copy codeCopy the code

Here is an example configuration:

// webpack.config.js const CDN_HOST = process.env.CDN_HOST; // CDN domain const CDN_PATH = process.env.cdn_path; // CDN path const ENV = process.env. // The current environment and so on const VERSION = process.env.version; Const getPublicPath = () => {// Some code here return '${CDN_HOST}/${CDN_PATH}/${ENV}/'; Const publicPath = process.env.node_env === 'production'? getPublicPath() : '.'; module.exports = { output: { filename: 'bundle.[name][contenthash:8].js', publicPath, }, plugins: [new HtmlWebpackPlugin()]} Copy the codeCopy the code

Note 1: We tend to deploy one set of code to multiple front-end environments, and we need to inject environment variables related to the current deployment at build time (such as staging, PROd, dev, pre, etc.) to dynamically build publicPath. Note 2: The dynamic publicPath is strictly segregated and converged by environment + release. A service front end used Hash to separate static resources in all environments. However, the file name and hash are suspected to conflict, but the file contents are different, resulting in an online accident. Therefore, it is recommended to make physical isolation of the products strictly. Note 3: publicPath explanation webpack.docschina.org/configurati… Note 4: Content-hash is used. For details about the differences between Contenthash, hash, and chunkhash in webPack, see: When contenthash is used, the hash of the entire file changes after a small module is added. The reason is that the module.id of Webpack increases by default based on the resolution order, which causes cache invalidation. You can set optimization.moduleids to ‘deterministic’. For details, please refer to webPack official document – Cache. Note: As for webpack configuration, it is only necessary to know about it in the early stage, and it is recommended for further study in the future.

  1. Static resource upload after buildCDNThe source station

CDN upload The source site usually invokes various client tools on the CLI to upload the CDN. In this case, the CDN upload depends on configuration authentication information, such as the Bucket Name/accessKey of the file storage and the FTP account password. These authentication information should not be written directly into the code, otherwise there may be a risk of accidents (think about why)! The third step is the Nginx layer reverse proxy transformation

Reverse proxy: A proxy server receives network requests, forwards the requests to an internal server, and returns the internal server, just like a primary host. Reverse proxy and forward proxy: Reverse proxy hides the real server, forward proxy hides the real client. See: Ramblings: How to explain to girlfriend what is reverse proxy?

Nginx can configure proxy forwarding by setting proxy_pass, for example

location ^~/static/ { proxy_pass $cdn; } Duplicate codeCopy the code

See nginx proxy_pass for details

Note: Students who recruit students or transfer to the front end of the client are advised to be familiar with and master the following.

Static resource organization summary

Finally, a review

  1. To make the most of the cache, place the page entry (HTML) to set the negotiation cache toJavaScript,CSSStatic resources are set to permanently strong cache.
  2. To solve the strong cache update problem, the file summary (hash) as a resource path (URL).
  3. In order to solve the problems caused by overwriting publishing, adoptname-hashRather thanquery-hashYou need to configure the organization modeWbpack 的 output.filename 为 contenthash 。
  4. In order to solve theNginxDirectory storage is too large + combinedCDNImproved access speed, adoptedNginx reverse proxy+ Upload static resourcesCDN.
  5. In order to uploadCDNWe need to construct dynamically according to the environmentpublicPath+ According to the environmentCDNUpload the directory and upload it.
  6. For dynamic constructionpublicPathAnd is inserted as part of the build processHTMLuseWebpack-HTML-PluginSuch plug-ins will compile good tapehash + publicPathTo insert static resources intoHTMLIn the.
  7. In order to ensure uploadsCDNSecurity, we need a mechanism to control uploadCDNThe secret key, rather than simply writing the secret key to code /DockerfileIn the plaintext file.

It’s layers of nesting dolls!

At this point, we have basically obtained a relatively complete front-end static resource organization scheme.

At this point you may have noticed that after the front-end static resource is deployed, there are also processes consumed by Nginx before users can access it.

Automated build

We have now explored a solution for static resource organization. Now look at the build process. We need to do these steps roughly each time we build:

  • Pull remote warehouse

  • Switch to the XX branch

  • Code security checks (optional), unit tests, and so on

  • Install NPM/YARN dependencies

    • Set up thenodeversion
    • Set up thenpm/yarn 源
    • Installation dependencies, etc.

  • Perform compilation & build

  • Product inspection (such as testing the size of packed JS files/pictures, product safety, etc., to ensure product quality, optional)

  • Manual card point (optional, if the Leader must approve to continue)

  • Package and upload to CDN

  • Automated testing (Optional, E2E)

  • Other steps are required

  • Notification of build completion

Among them, the problems facing us are:

  • In what environment is the build performed?
  • How do I ensure that the deployment environment is the same for each build?
  • Who triggers the build?
  • How do I manage the uploads described aboveCDNAnd other keys (no additional cost, guarantee security, guarantee build upload reliability)?
  • How do you automate the build trigger & automate the above steps?

If it is executed manually every time, it is estimated that the compilation will be wrapped on release day, and it is more likely to cause problems, such as missing a step or the wrong order.

  • How to increase build rate?
  • How to inform r&d students that the construction is completed?

Soul torture no!

To solve the above problems, the industry has several solutions:

  • Ensure environment consistency: Docker
  • Build by process: Jenkins
  • Automated build triggers: Gitlab Webhook notifications
  • Start building notifications: Rely on account system to get through + Gitlab Webhook
  • Notification of completion of construction: rely on account system to get through

The general implementation in the industry is generally Jenkins + Docker + GitlabWebHook. For example, the following are some practices:

IDeploy – A continuous delivery platform for front end teams to build deployment engineering

There are other issues: node_modules, the heaviest matter in the universe, is too slow to install? How to Improve Build Speed The above mentioned Build systems & solutions are relatively complete in every major company, which varies from company to company but is generally similar, so this step will be skipped in this article.

Front-end release services – pre-release environment, version management (second level rollback), small flow, gray scale, AB test

Assuming our static resource organization is complete, automated build deployment is done, and Nginx reverse proxy is in place, our site is finally live for the first time. But what about the second and third launches? It’s extremely risky to send regression tests directly to production, but you can’t locally deploy the front-end test environment to connect to the back-end production library (think about why), so we need a pre-release (Pre) environment that is consistent with the production environment except for non-testers! This is where the first requirement comes in, pre-shipping the environment functionality. If we a function is New Year’s day zero point release, New Year’s day guard in front of the server point release? If the NPM fails to pull dependencies and the build fails, or if there are bugs when it goes live, it will be cool. Or, as the front-end projects pile up over time, node_modules become more massive than the total mass of the galaxy, often taking more than 20 minutes or more to build. One day, after we launched the new function, we found a fatal obstructive bug, and the payment will be automatically refunded by 1.5 times! Want to roll back the version now? Well, just wait and watch it compile. This time is really time is money, recompile slowly the company will go bankrupt. Is there a way to roll back the version as soon as a problem is found? And the rollback operation, the rollback students should not log into the server to do the operation (think why?). . This is where the second requirement comes in, version management capabilities. Static resources can be brought online in advance, each historical version needs to be kept, and versions can be switched instantly, and the switching process should not be logged into the server (think about why). Secondly, it is assumed that PM is constantly optimizing functions and wants to scale some users first, or wants to do some AB tests, such as promoting Fujian cuisine to Users in Guangdong and bobo chicken to users in Chongqing. At this time, we have two plans. The first plan is to package the chicken and Fujian cuisine into a code product, and then switch according to the region during the runtime. But pretty soon you’re going to have chicken in your code, cold chicken in your code, hot chicken in your code, mom rabbit head in your code, beef in your code, Wolf teeth in your code, potatoes in your code, hokkien food in your code, and so on. Besides, I would be the first to disagree with the combination of hot and cold kebabs, which is the blasphemy of food! So plan one is not desirable.

In practice, it is often the case that two completely incompatible changes are run online at the same time as the AB test.

The second plan is that we pack hot pot skewers and cold pot skewers separately, so that hot pot does not make cold pot. Some mechanisms were also designed. For example, students carrying banana candy (cookie) gave beef pot to cross feet, students speaking Hong Kong dialect gave Fujian cuisine pot, and students from Sichuan region randomly gave dry pot, soup pot and fish pot to hot pot. Isn’t that fun? As you can easily see, this mechanism is extremely variable, with a high probability of change. Do we have to log in the server to adjust the ratio of dry pot and fish hot pot every time we want to adjust it? What if I run out of dry POTS one day but don’t bring my computer home? At this time encountered the third demand, at any time to adjust the small flow Test, AB-test Test, gray on-line and other functions. To sum up, in order to meet complex online requirements, the deployment level generally needs: pre-release environment, version management, small traffic, gray scale, AB testing and other functions.

Processing of static resources

As mentioned above, after the front-end static resources are deployed to the CDN, there is a Nginx reverse proxy to do the processing of forwarding. In fact, people often need to do more work with static resources to solve various deployment problems or improve performance. For example, in order to improve the first-screen performance of some Web applications, a common way is to direct HTML through BFF layer or through the back end, and inject some information in the process, such as userInfo, user permission information, gray information, etc., so as to greatly reduce the front-end login research and development cost & reduce the first-screen time. The following is a brief process for pulling out HTML from the back end.

The main process is that the HTML constructed for the front end contains a number of template variables. After receiving the request, the back end converts cookies into user information through various Proxy layers, loads index.html from CDN according to version configuration, and replaces template variables with user information by using template engine. Finally, you spit back to the browser HTML that already contains the user’s information!

Pre environment, gray line common implementation

As mentioned earlier, our static resources are non-overwrite releases, and there are several version static resources online after multiple deployments. The idea of Pre environment/grayscale online is: through a certain mechanism, let specific users access specific static resource version, so as to achieve access to Pre/ grayscale online ability.

Scheme 1: Nginx layer dynamic forwarding

A common Pre mechanism is that after multiple versions of static resources are deployed, developers use browser plug-ins such as ModHeader to carry a specific Header (such as xx-env= Pre) in the request, consume the Header in the Nginx layer and dynamically forward the Header to the static resources of the corresponding environment. Access the Pre environment. At this point, all environments are production environments, with the exception of static resources that are specific versions, and the range of variables can be kept to a minimum.

The general process is shown as follows:

Nginx can set forwarding by configuring rewrite, as shown below.

See the rewrite directive in nginx configuration for details

location /example { rewrite ^ $cdn/$http_x_xx_env/index.html break; proxy_pass $cdn/prod/index.html; } # $http_x_xx_env represents copied code from the defined Request Header field xx_envCopy the code

Note: As for Nginx, it is recommended to be familiar with it and master it later.

The advantages of this scheme are simple configuration and high efficiency, suitable for engineers. Disadvantages: Each user needs to manually configure the Header, which is not applicable to mobile terminals. In addition, specific users cannot passively and accurately access a certain version. For example, configuring the Header for PM and KP users costs too much. Similarly, the Nginx layer can also be processed according to some other rules to achieve grayscale on-line ability. Such as through a certain number of random rewrite, to achieve a small range of random gray scale. Obtain UA and rewrite to achieve browser-directed grayscale. Obtain regional information through Nginx GeoIP to achieve gray scale by region. However, the gray scale scheme above is complicated in configuration, and the gray scale ratio/range is often configured too much. Therefore, operation and maintenance need to log in to the production server for modification every time it goes online, which is prone to various accidents. Therefore, it is not recommended to use, only for broadening the idea.

Scheme 2 Dynamic configuration + server forwarding

However, Pre environment or gray scale often needs to accurately locate some specific groups, such as specific PM, HR, remote error reporting specific users, KP users or even a certain department to open Pre environment. The above students lack engineering background/are busy/access through mobile terminal. In this case, it is no longer applicable to modify the Header. So we’re still looking for a mechanism that makes it easy to adjust the Pre/ grayscale range at any time without having to re-launch. Since you need to Orient by user dimension, you rely on the back end to handle it for you.

In order to be able to adjust the gray/Pre strategy anytime and anywhere, rather than relying on the adjustment of the code issued online, the concept of the configuration center is introduced at this time.

Configuration center: it is generally an independent platform/SDK, providing dynamic configuration management solutions, providing functions such as configuration management, version management, permission management, gray publishing and so on. Backend applications are consumed through interfaces, so the configuration center is decoupled from the backend and the configuration can be modified at any time rather than reissued. The configuration center typically configures a JSON object. Manual maintenance of the central JSON object may cause problems. Therefore, robots are added to reduce the probability of errors.

The following is the flow chart of dependency configuration center + server forwarding:

The main process is:

  1. Front-end siege lion students deploy multiple versions of static resources to the CDN (question? How do I manage multiple versions of static resources? .
  2. After receiving the request, the backend converts cookies into user information through various Proxy layers.
  3. The backend reads the configuration center data, determines which environment the user accesses according to the user information, and loads the specific environment index. HTML
  4. The back end returns the processed index.html to the browser
  5. If you need to add the specific KP equivalent to learning Pre list, the students only need to call the robot /Bot, modify the configuration center, it will take effect.

Note that under the above architecture, if some problems that are difficult to troubleshoot occur to an online user, a specific version can be released. After modification in the configuration center, users can access the page of a specific version, thus simplifying the troubleshooting process.

At this point, some small traffic configuration, AB experiment, version management can also be implemented through this scheme. The advantages of this scheme: it can be adjusted at any time, and it can also take effect on the mobile terminal without the back-end version. Disadvantages of the scheme:

  1. Strong binding to the server (requiring user information, unavoidable).
  2. Every time you have to go fromCDNloadingHTML, there is a certain performance waste. But if the cacheHTML, the release link should also notify the server, increasing the overall complexity.
  3. If you considerCDNFault, the server doesCDNDemotion adds complexity.
  4. Version management/small traffic are common requirements, and every back-end application of this solution needs to be developed or connected.
  5. The common configuration center is generallyJSONConfiguration, relatively simple, and the release of the multi-environment can not be associated, rely on artificial configuration, there is the risk of error (such as the releasev1.2501, the configuration center manual configuration error changed tov1.2051).

Front-end publishing service implementation and design

Some students may misunderstand the version management of online products and add version management to the code (such as manually/automatically tagging after version release), which can be met by version release deployment when needed. However, as mentioned above, the flexibility of version management through source code is poor, unable to do one-button & second-level version switching, which does not meet the needs of the changeable & complex commercial environment. So how do you do versioning? The answer is deep processing & management of constructed products. At the same time, versioning/small traffic is a common business requirement for front-end deployment and should be separated from business back-end services. Therefore, a new public service is proposed for front-end deployment only, called Page Server. For specific index.html file management & undertake Nginx traffic or business back-end traffic, etc. At the same time, in view of the frequent adjustment of version management and small traffic policy, we should not log in to the Server every time we adjust, so we need a new service & interface for operating version management, adjusting small traffic and other information, and synchronization with the Page Server mentioned above. Here we call this service Page Config Web. Our Page Server may have many instances deployed on multiple clusters to meet the requirements of multinational deployment, multi-department project deployment, and so on. Ideally, Page Config Web **** also undertakes the creation, management and configuration of PageServer. So PageConfigWeb and PageServer are 1-to-n relationships (or M-to-N for cross-border deployments, etc.). At the same time, we have a front-end project may have multiple sets of front-end environment, PageSever in a fixed cluster is a public facility, these environments can theoretically be carried by one or more PageServer. So a PageServer has a one-to-one or one-to-n relationship with multiple front-end environments. At this point, for traffic coming from Nginx, we need a mechanism to distinguish which environment instance the traffic belongs to, such as through URLS, which we can call routing. Finally, to ensure correctness and automation of the above services, after the build deployment (new version) is complete, synchronize to the above two services to ensure correct version management.

Finally, the general flow chart is as follows:

Oversized graph warning

In essence, there is a common intermediary service that is deployed on multiple clusters, deeply tied to the build publishing process, that is used to take HTML traffic and set small traffic rules, versions, and so on through the Web site to meet the changing requirements of going live. PageServer, while hosting HTML services, can do some other things, such as:

  1. SSR
  2. CDN degradation is used to replace static resources with available CDN sites directly out of HTML when CDN is abnormal.
  3. 404 processing
  4. Backpocket page (e.g. when the service fails and cannot be fixed in a short time)
  5. Template rendering (for example, to do template substitution, replace query to template medium)
  6. Special global processing, such as global style injection to global page graying

And so on and so on.

PageConfig Web and PageServer have all version information after construction, theoretically can cache each version of THE HTML file, and in order to optimize performance, PageServer can cache the latest full version of the HTML file in memory, maximize the response speed. The remaining versions are stored in caches such as Redis.

The following uses an official release V.1.0.2502 and the rollback process as an example:

  1. Code merges to trigger automatic build, and build products are organized by environment (ENV) + version (env) + version (version) + name-hash, and uploaded to CDN.
  2. After the build is complete, the build script informs the students of the siege Lion and synchronizes PageServer and PageConfig Web service with the new version V.1.0.2502.
  3. After receiving the notification, go to PageConfig Web site to publish the new version V.1.0.2502 (PRE), and configure the PRE environment small traffic rules for this version.xx-env = pre. At this point, the version can only be accessed by setting a specific Header.
  4. If Nginx forwards the PRE directly, siege Lion accesses the PRE version by setting the Header.
  5. If the server is used to forward the PRE version, Siege Lion can configure the PRE whitelist in the configuration center to allow users to access the PRE version.
  6. After PRE acceptance is complete, Siege Lion logs in to PageConfig Web site and releases official version V.1.0.2502 (without small traffic information). It takes effect immediately.
  7. After the effect of the online regression, found a bug, siege lion immediately log in to the PageConfig Web site, rollback to the previous version v.1.0.2501. It takes effect immediately.

Summary of deployment

Static resource organization section

  1. To maximize cache utilization, set pages (HTML) to negotiated cache, and JavaScript, CSS, and so on to permanently strong cache.
  2. To solve the problem of strong cache updates, file digests (hashes) are constructed as part of resource paths (urls).
  3. To solve the problem caused by overwriting publishing, use name-hash instead of query-hash to organize the webpack output-filename to contenthash.
  4. In order to solve the problem of Nginx directory storage is too large + combined with CDN to improve access speed, Nginx reverse proxy + upload static resources to CDN.
  5. To upload the CDN, we need to dynamically construct publicPath according to the environment + construct CDN upload directory according to the environment and upload.
  6. To construct publicPath dynamically and insert it into HTML along with the build process, plug-ins such as webpack-html-plugin are used to insert compiled static resources with hash + publicPath into HTML.
  7. In order to ensure the security of CDN uploading, we need a mechanism to control the uploading of CDN secret keys, rather than simply writing the secret keys to plain files such as code/Dockerfile.

Automated deployment

In order to improve the deployment efficiency and 100% avoid deployment errors, it is necessary to design and build an automatic deployment platform, with Docker and others to ensure the consistency of the environment and Jenkins and others to ensure the series of the construction process. Use es-Build etc to improve build efficiency.

Front-end deployment & Static resource processing

The following principles/requirements can be summarized for front-end deployment:

  1. After a build is published, it should not be overwritten.
  2. After a build is published, static resources should be permanently stored on the server /CDN, i.e. readable only.
  3. In the static resource organization, each version should be stored by folder to achieve resource convergence. If want to delete really so, can delete by version. (such as a version of the code leak)
// webpack.config.js const CDN_HOST = process.env.CDN_HOST; // CDN domain const CDN_PATH = process.env.cdn_path "; // CDN path const ENV = process.env. // The current environment and so on const VERSION = process.env.version; Const getPublicPath = () => {// Some code here return '${CDN_HOST}/${CDN_PATH}/${ENV}/${VERSION}/'; Exports = {output: {filename: 'bundle.[name][contenthash].js', publicPath: {output: {filename: 'bundle. GetPublicPath (),}, plugins: [new HtmlWebpackPlugin()]} Copy codeCopy the code

Therefore, the version field should be added to publicPath

  1. The publishing process should be automated and developers should not have direct access to the server.
  2. During version switching, do not touch the server.
  3. The version switchover takes effect in seconds. (For example, switching from V0.2 to V0.3 takes effect immediately).
  4. Online needs to be able to take effect multiple versions at the same time, meet the AB test, gray, PRE environment and other small traffic requirements.

All the above requirements are relatively complex and changeable. In order to cope with complex online requirements, static resources can be deeply processed, such as direct HTML output through the server, PRE PRE by user through the configuration center and so on.

Front-end publishing service

In the face of complex commercial needs, convenient multi-front-end business to achieve version management, gray scale, PRE, AB testing and other small flow functions, we designed an intermediate service PageConfig Web & PageServer, and Nginx and a variety of backend combined to achieve the ability of configuration immediately effective.

Part of the answer to soul torture

Q: What process does front-end code go through from TSX/JSX to deployment and access by users? A: Experienced local development, remote construction package deployment, security check, upload CDN, Nginx to do traffic forwarding, static resources to do some processing and so on. Q: Most of you are probably familiar with strong/negotiated caching. What caching strategies should be used for various front-end artifacts (HTML, JS, CSS, IMAGES, etc.)? And why?

  • If a negotiated cache is used, but static resources are not updated frequently, how can you avoid wasted requests during the negotiation process?
  • How can static resources be updated if strong caching is used?

A: Negotiation cache is used for HTML, strong cache is used for static resources, name-hash is used for static resource updates. Q: How should the supporting, front-end static resources be organized? A: With Webpack’s webpack_html-plugin & configuration output publicPath etc. Q: How does the accompanying, automated build & deployment process integrate with the CDN? A: After the automated build is packaged, the product is transferred to the CDN of the corresponding environment URL. Q: How do I prevent the front end from going online and affecting users who have not refreshed the page? A: Use name-hash to organize static resources and then HTML. Q: How can I do a second rollback of a new release that has a blocking bug instead of waiting 20 minutes or more to deploy again? A: HTML files are stored on THE CDN in A non-overwriting way, front-end publishing service is set up, and HTML is cached and processed according to version. When rollback is required, change the publishing service HTMl pointing. Q: When the CDN domain name suddenly hangs up, how to achieve the second-level CDN downgrade repair instead of redeploying all services once again? A1: Transfer static resources to multiple CDNS and develop an SDK to load Script for integration into HTML. When CDN resources fail to be loaded, degrade the CDN domain name gradually. A2: Add HTML text processing in the front-end publishing service. For example, add CDN domain name replacement. In case of an exception, click the button to set it in the publishing service. Q: How to realize a pre-delivery environment, except the front-end resources are all online environment, control variables in the front-end environment? A: Do processing to static resources, do small traffic to HTML entry. Q: How can we facilitate AB testing in the deployment process? Q: How do you implement A set of front-end code and release it into multiple sets of environment artifacts? A: Use environment variables to inject the current environment, CDN, CDN_HOST, Version, etc., into environment variables, and then consume & upload products to different CDN during construction.

other

If you want to learn more about front-end deployment, here are some tips.

  1. Learn load balancing (Level: Understanding). Learn the principles of load balancing and how to configure it. For example, see the large Web architecture series: Load balancing details
  2. Learn more about HTTP (Proficiency), such as common status codes, common headers and their in-depth application, strong/negotiated caching, new features of HTTP2, and so on. In particular, HTTP 1.1 and HTTP 2.0. Recommended book: Illustrated HTTP The definitive guide to HTTP
  3. In-depth study of front-end engineering (Requirement: Proficient) A. Understand what front-end engineering can do, such as Front-end engineering: System Design and Practice B. Master common practices of front-end engineers & practical operations c. Deep learning Webpack Official documentation
  4. Learn various solutions for processing front-end static resources (Requirement: Master)
  5. Deep learning browser principle (requirements: proficient) some information: from the browser process to JS single thread, JS running mechanism of the most comprehensive combing