Why do we need Rendertron?

Traditional Web pages are usually rendered on the server side. However, with the popularity of SPA (Single-page Application), especially front-end frameworks represented by React, Vue and Angular, more and more Web apps use client-side rendering.

There are many advantages to using client-side rendering, such as saving backend resources, partial refreshes, and separation of front and back ends, but there are also some challenges, such as the SEO issues addressed in this article.

For the page rendered by the server, the server can directly return the content in the form of HTML, and the search engine crawler can easily get the page content, while for the application rendered by the client, the client must execute the Javascript returned by the server to get the correct page content. Currently, most search engines do not support Javascript, with the exception of Google and Bing (with some limitations), which makes it impossible to get the correct web content.

Google’s Rendertron is a tool designed to solve this scenario. Using Rendertron, SPA can also be used to crawl rendered content by search engines that do not support Javascript execution. This works by using Headless Chrome to execute Javascript in memory and return the content to the client when it is complete.

Rendertron is introduced

Rendertron is typically deployed as a standalone HTTP service, and then middleware provided by Google is configured for the Web application framework or routing rules are added to the reverse proxy so that when UA of search engine crawlers is detected, Requests can be brokered to the Rendertron service.

Rendertron provides two main apis, Render and Screenshot. Render is used to Render site content, Screenshot is used to Screenshot site content. The Render interface is used in SEO scenarios.

For example, when the client requests our website, we search the line according to the request header UserAgent and find the keyword Baiduspider/2.0, which can be identified as the current client is a Baidu crawler, and then find the keyword Mobile in UserAgent. It can be identified that the crawler is doing mobile content grab. Through the above judgment, this request can be agent/render/https://www.aliyun.com/?mobile Rendertron service routing, let Rendertron help execute Javascript within the web page, The final content is returned to the search engine crawler.

Effect in

Google has an example render-Tron.appspot.com/, which allows you to experience the effects directly.

We also provide the deployment on function calculation example: renderton. Mofangdegisn. Cn

System architecture

Based on function calculation, our service architecture is as follows:

The performance test

Here we choose ali Cloud performance test PTS service pressure test.

The test configuration is as follows:

We configured 100 concurrency, tested for 6 minutes, and pressed the concurrency per minute according to the rule of 20% increase.

We want to test the url site for: renderton mofangdegisn. Cn/render/HTTP…

The url tells Rendertron to request the content of www.example.com/ and return the render results.

The test overview is as follows:

As you can see from the overview above, the minimum latency is 1106ms due to network requests from functions to www.example.com/, 99% of requests can be completed at 2011ms and 90% at 1347ms, 75% of requests can be completed at 1201ms and 50% at 1156ms. We increase the concurrency by 20% per minute. When the concurrency increases, the function calculation will encounter cold start, and the maximum time of cold start is 32261ms (the influence of cold start can be alleviated or completely eliminated by preheating, reservation and other methods).

In an unoptimized scenario, we achieved a TPS of 44.91, which is definitely enough for most sites.

Pressure measurement details are as follows:

The position indicated by the arrow above indicates a sudden increase in concurrency. The automatic expansion of the function encounters some cold starts, and when the expansion is complete, subsequent requests are very smooth.

Deployment steps

Deploying Rendertron to a traditional ECS or physical machine as a production service was not an easy task. In addition to the installation configuration required by Rendertron itself, there is also the need to consider how to expand as traffic increases, and to configure services such as setting up reverse proxies or load balancing to go along with it.

Now, let’s look at how functional computation solves these problems.

Refer to the link

  1. Fun Installation Tutorial You can download the binary version of Fun directly here, unzip it and use it.
  2. Docker installation tutorial: local installation depends on the use of Docker, you can directly download Docker here.

1. Clone Project:

git clone https://github.com/GoogleChrome/rendertron.git
Copy the code

PS: This is based directly on the official project, rather than providing a sample template, to demonstrate how to smoothly migrate Rendertron to function calculations, and also to update to the latest version as soon as the official version is updated.

2. Create template.yml configuration file:

Template. yml is Fun’s default description file. Resources described in this description file can be deployed to the cloud using Fun Deploy with one click.

For example, our template below declares a service named Rendertron and a function named Rendertron.

Function is the basic unit of function computing system scheduling and code execution. Our Rendertron project can run in function, and service is the unit of management function computing resource management. A service can contain multiple functions.

For starters, you can simply place the following template in the project root directory and name it template.yml.

ROSTemplateFormatVersion: '2015-09-01'
Transform: 'Aliyun::Serverless-2018-04-03'
Resources:
  Rendertron: Declare a service named Rendertron
    Type: 'Aliyun::Serverless::Service'
    Properties:
      Description: This is Rendertron service
    rendertron: Declare a function named rendertron
      Type: 'Aliyun::Serverless::Function'
      Properties:
        Handler: index.handler
        Runtime: custom # Runtime, we use custom
        Timeout: 60
        MemorySize: 1024
        CodeUri: . /
      Events:
        httpTrigger: Add HTTP triggers
          Type: HTTP
          Properties:
            AuthType: ANONYMOUS
            Methods:
              - GET
              - POST
              - PUT
           
  renderton.mofangdegisn.cn: Add a custom domain name
    Type: 'Aliyun::Serverless::CustomDomain'
    Properties:
      Protocol: HTTP
      RouteConfig:
        routes:
          / * :
            ServiceName: Rendertron
            FunctionName: rendertron
Copy the code

3. Create the bootstrap file

Next, create a file called bootstrap in the project root directory. This file tells the function to calculate how to start rendertron. The file looks like this:

#! /usr/bin/env bash
PORT=9000 HOST=0.0.0.0 npm run start
Copy the code

4. Install dependencies & build projects

Fun install -d installs puppeteer dependencies that rendertron contains in addition to NPM dependencies. The required APT dependencies for Puppeteer are automatically installed. See this article for more details:

fun install -d
Copy the code

Then compile the project using the official introduction of NPM run Build:

npm run build
Copy the code

5. Run Rendertron locally

Don’t need to modify the original program code, we can directly through fun. Local start renderton mofangdegisn. Cn locally will function start, and then through the browser to access.

fun local start renderton.mofangdegisn.cn
Copy the code

The demo effect is as follows:

6. One-click deployment

When running locally and debugging verifies that there are no problems, you can consider deploying online. Before deployment, replace the domain name in template.yml with your own.

Here are the steps: If the AccountId of Aliyun is 12345, then update your domain name CNAME to 12345.cn-shanghai.fc.aliyuncs.com and then update your domain name to template.yml. Run the fun deploy command. See this document for more details.

Finally, use Fun Deploy for one-click deployment.

conclusion

Rendertron + function calculations can be used to quickly build a Headless Chrome rendering solution that can be used directly in production to help websites do BETTER SEO.

“Alibaba Cloud originators pay close attention to technical fields such as microservice, Serverless, container and Service Mesh, focus on cloud native popular technology trends and large-scale implementation of cloud native, and become the technical circle that knows most about cloud native developers.”