Odd technical guidelines
This article describes my trip to the front end of containerized deployment. And solve the following problems: Node and Modules cache; Mirror volume is too large; The number of mirrors is large and the online process is complicated.
This article was first published on the public account “Qiwu Weekly”. The author of this article is Qiwu Front-end development engineer Huang Xiaolu, who is also a member of W3C performance Working group.
This article describes my trip to the front end of containerized deployment. And solved the following problems:
-
The node modules cache
-
Oversize mirror image
-
The number of mirrors is large and the online process is complicated
This article requires readers to have a certain container foundation, master the basic operation of Docker, understand the Pod principle of K8s. If you don’t know, please go to the end of the article and read the resources.
A front-end project was recently deployed to the company’s container services.
It started like this: We had a Jenkins set deployed on a server for front-end project integration. A month ago, security sent us an email pointing out a vulnerability in our Jenkins plugin. So we spent two days backing up the configuration, applying Deploy keys for each project for the server, and upgrading Jenkins. Colleagues couldn’t get online quickly and had to manually copy to an online server.
The author reflected that the existing process has the following problems:
-
This integration service does not do disaster preparedness, plus there are some version compatibility issues to resolve during the upgrade. Once down, affect production efficiency
-
Each time you deploy a new project, you need to apply for a machine, install a set of environment dependencies, and add access to Jenkins’ machine
-
When using Jenkins, you need to specify the target machine address in the script. If you want to add a new server, you have to repeat steps 2 and 3
In the long run, we need integrated services that are more stable and automated (and maintained by more specialized people). The company’s cloud platform provides container services based on Kubernetes (K8s for short) and can be continuously integrated. It fits our needs. Start your new deployment.
The first deployment
Since the project is currently completely separated from the back end, it is only necessary to compile the static resources from the source code, and then use Nginx to associate the front-end domain name with the static resources. So I created two images based on Node and Nginx. Run the two mirrored containers in a Pod, and let the two containers read and write static resource directories through shared space. As shown below:
The Node image is built in the continuous integration process of the cloud platform. Because the cloud platform has applied for Webhook in Gitlab, the build process is automatically triggered when the code is submitted. Nginx images are built locally and then uploaded to the cloud.
General process of cloud Node image construction:
The contents of node.dockerfile:
FROM $NODE_BASE_IMAGE
WORKDIR /site
WORKDIR /site-build
ADD ./ /site-build/
RUN npm install && \
npm run build && \
mv /site-build/dist /site/ && \
rm -rf /site-build
ENTRYPOINT $MOVE_DIST_TO_SHARED_FOLDER_AND__KEEP_CONTAINER_ALIVE
Copy the code
First use FROM to specify a Node base image, specify the working directory, add the project file to the image, run NPM install and NPM run build, copy the result to the /site directory. And remove /site-build (only the compiled files are left, the image size is smaller).
Finally, declare in ENTRYPOINT that to start the container, copy the compiled folder to the shared folder and run a command to keep the container from exiting. (Any command, just don’t quit. The container restarts continuously because the exit auto-restart function is enabled online.
Nginx.dockerfile, which builds Nginx images locally, is much simpler:
FROM $NGINX_BASE_IMAGE
ADD ./nginx.conf /$NGINX_CONF_INCLUDE_PATH
EXPOSE 80
Copy the code
This configuration first specifies the base image and then adds the local Nginx configuration file to the corresponding Nginx include directory in the image. Because Nginx images are built locally and caching is enabled by default, the build process can be extremely fast as long as the ADD files remain the same and the various commands remain the same.
After the image is built and uploaded, configure the shared folder of the container and the path for mounting nginx logs on the cloud platform. Select the two images and publish them. The initial deployment is complete.
But there are several obvious problems:
-
Continuous integration of cloud platforms has no caching. This means that NPM install is executed every time a Node image is built. Obviously this is not necessary. If node_modules can be cached, there is no need for NPM install as long as third-party dependencies remain the same.
-
The mirror image is too large. Two 2GB+ base images. Uploading is slow and takes up unnecessary disk space.
-
The container running Node must remain alive. The container only needs to write the dist directory to the shared space at startup.
Optimization 1: Cache node_modules
The solution is to package node_modules into the base image and rebuild only when the local node_modules changes. From there, a second image is built to provide the compiled dist directory. The following issues need to be addressed:
-
The system automatically builds a new base image after detecting third-party dependency changes and uploads the image to the cloud platform.
-
Automatically writes the new base image address to the FROM field of Node.dockerfile.
-
Automatically submit the updated Node.dockerfile to continuous Integration to build the new image online.
Git hooks are a natural choice for automation. Because the image needs to be updated only before it goes live, NPM version hooks are used (none of the other hooks are appropriate). Add version and postVersion hooks to the scripts of package.json to automatically detect build and push code, respectively. Just run NPM Version (Major/Minor/Patch) to automatically detect third-party dependency changes and build the image.
"scripts": {
"version": "sh auto-build-node-base-image.sh",
"postversion": "git push && git push --tags"
}
Copy the code
How do I detect node_modules changes?
The first attempt is to detect changes in package.json (with git’s commit log to compare the last change). Package. json dependencies correspond to dependencies of package.json. If package.json has a new commit record, dependencies change. However, each execution of NPM version changes the version field in package.json and also commits package.json, so it is not accurate.
The second thought is package-lock.json. Changes in third-party dependencies are also reflected in package-lock.json. Package-lock. json also contains the Version field. Forget about it.
Yarn came to mind. Look at yarn.lock, which does not contain the version information in package.json. Yarn. lock changes only if the dependency changes. Perfect. I just abandoned NPM and went to YARN.
The optimized process is roughly as follows:
The original cloud build took an average of 400 seconds, but after optimization, it took an average of 210 seconds, saving half the time.
Optimization 2: Reduce the mirror volume
The original Node image and Nginx image size is 2GB+, can be said to be very bloated. The really useful static files (the dist directory) add up to less than 10MB. So you need to replace it with a smaller base image.
The original base image used was the CentOS operating system, which is large in size. What was the most popular version of Linux in the container era? Alpine of course. The reason is that it’s small enough. How small? About 5MB.
So in the first step of optimization, by the way, the local Node base image is changed to Alpine, build the image and then install YARN (command: RUN apk add –no-cache YARN).
The original locally built Node image (node_modules installed) was 2.53GB, reduced to 668MB after replacing the base image.
A Node image built locally, compiled in the cloud to generate the dist directory, and then removed from the node_modules directory, becomes even smaller.
Optimization 3: Multi-stage build
Moving on to the third question, Node containers don’t need to live. All we need for our site is Nginx+ configuration file + static resource directory.
The multi-stage build feature of Docker is used in this case. It allows multiple build steps to be written to a Dockerfile. This is equivalent to executing multiple steps in a mirror build, depending on the files and instructions generated in the last step.
The first phase generates the DIST directory, and the second provides the Nginx configuration and static resources. The construction process is as follows:
End result:
-
Image size reduced to 25MB (originally 2 images 2GB+)
-
Run only one container in a Pod and remove the shared space.
-
Continuous integration build time averaged 170 seconds (previously averaged 400 seconds)
Overall, the optimization effect is more obvious. Hope to provide some reference for you to do container.
Reference documentation
-
10 minutes to understand Docker and K8S
-
Docker Tutorial
-
Use multi-stage builds
-
How to Deploy front-end Applications efficiently with Docker (Shanyue)
The latest activity
360 Internet Technology Training Camp 15th session
— Large-scale service architecture practice
Identify the qr code below or click to read the original text to register now
World of you when not
Just be your shoulders
There is no
360 official technical official account
Technology of dry goods | | hand information activities
Empty,
Click “Read the original” to sign up now