Antecedents feed
I bought a cloud server in Ali cloud at almost a white prostitute price before, but it is about to expire. If the fee is renewed according to this configuration, the cost is relatively high, so I bought a cloud server in Tencent cloud as a new user.
With the new server, the next problem is how to migrate the services and code previously deployed in Ali Cloud to the newly purchased Tencent cloud.
Node, Nginx, Jenkins and so on are installed in the traditional way on the old server. Although relatively convenient project release process is finally realized, its installation and configuration are quite tedious. Due to the existence of Docker, it can make all this relatively simple, so I want to use Docker on the new server to deploy a complete front-end project release system.
This deployment uses this site as an example: Ssr.mimei.net.cn, this is a VUE server-side rendering project, simple function, but contains node, PHP services, to make it run normally, need to make git, nodeJS, PHP, mysql, nginx, Jenkins and other software work together. Can be said to be a relatively complete project, perfect for a demonstration.
The main content of this article
In general, to complete the deployment of the site, the release of the code, need to be divided into the following parts:
- Setting up Git Server
- Generate the Mysql container and migrate the database
- Generating a PHP container
- Generate the NodeJS container
- Generate the Nginx container and configure it
- Generate Jenkins container and initialize data
- Use Docker-compose to manage multiple containers
- Configure Jenkins to automate project release
The above projects are followed by a complete front-end project release system.
Release process
Finally, submit the local code to remote (Git server or Github) and log in Jenkins to manually publish the project.
The release flow chart is as follows:
Graph LR modify local code --> push to remote --> Jenkins operation publish --> Publish completeconvention
Since code migration is involved, there is a convention here
Old machine – refers to my Aliyun machine, which is about to expire
New machine – Tencent cloud machine, newly bought; Data will be migrated from the old machine and new services will be deployed here.
other
About this site https://ssr.mimei.net.cn, about Vue SSR, I also wrote an article before, looking back, it has been nearly 3 years ~
This article will cover a little bit of Linux, Nginx, and Docker, so if you’re not familiar with it, check out other blogs.
Let’s start with the formal content
Setting up Git Server
One question to consider at the start of a project is: Where does the code live?
Usually the following two directions:
- Free hosting of open source remote repository, such as: Github, code cloud, etc
- Build your own Git server
To provide a more comprehensive demonstration, I have stored the front-end vue code in Guthub, and the server API code (PHP) in my own Git server.
Skip some guthub operations.
Start creating Git services on your server and log in to your new machine:
Creating a Git user
adduser git Create a git user to run git services
passwd git # Set user password. Later pull,push and other operations will be performed by password
Git users are not allowed to log in to the shell. This can be done by editing the /etc/passwd file. Find a line like the following:
git:x:1001:1001:,,,:/home/git:/bin/bash
# is changed to:
git:x:1001:1001:,,,:/home/git:/usr/bin/git-shell
Git users can only use git to perform resource operations, not shell login
Copy the codeInstall git
yum install git # install git
mkdir /opt/git && cd /opt/git
Create a git directory where all subsequent git project files will be stored
Go to /opt/git
# To facilitate management, I created a sub-directory mimei.net under /opt/git to store the files related to this project
git init --bare mimei.api.git
--bare creates a bare repository with no current working directory
chown -R git:git mimei.api.git
Change owner to git
# at this point, we created a naked warehouse, his address is: [email protected]:/opt/git/mimei.net/mimei.api.git
Copy the codeThe code to access
Going back to the old machine, the original API code exists here:
Enter the API project (PHP code) directory
git init # Initial Huagit project
git add .
git commit -m 'init'Git remote add origin git@http://82.156.7.129:/opt/git/mimei.net/mimei.api.gitAdd remote address
git push origin master Push code to a remote machine
Enter the password of the git user set before, and push the code successfully
[email protected]'s password: enumerate object: 155, done. Object count: 100% (155/155), done. Use 4 threads to compress the compressed object: 100% (122/122), complete. Written to the object: 100% (155/155), 145.70 KiB | 36.42 MiB/s, completed. A total of 155 (18) difference, 155 (18) difference To reuse 82.156.7.129:/opt/git/mimei.net/mimei.api.git * [new branch] master - > master # at this point, We've already put the server code on our own machine, so we can pull it off whenever we have a networkCopy the codesummary
At this point, all the necessary code for the project is ready:
VUE code: https://github.com/wfzong/mimei.web.ssr.git, making regular operations here is no longer here.
Server-side code (PHP) : [email protected]:/opt/git/mimei.net/mimei.api.git
Now it’s up to you to make the project run.
Generate the Mysql container and migrate the database
The backup data
Install mysql. Log in to the old machine and back up mysql data
# https://www.jianshu.com/p/5ec717d629aa
Server version: 5.5.64-MariaDB MariaDB Server
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| #mysql50#.git |
| SZBPY |
| limpid |
| mysql |
| performance_schema |
| phpmyadmin |
| wfzong |
| wordpress |
+--------------------+
# backup database
mysqldump -u root -p --quick --databases DB1 DB2 > /data/db.sql
Copy the codeWhen the backup is complete, copy the data from the old machine to the new machine
Copy the data to the target machineSCP/data/db. SQL [email protected]: / data/fuzong. Wang /Copy the codeGenerated container
Run the mysql service on the new machine
docker run \ -it \ -d \ --rm \ --name minei.net-mysql \ -v /data/fuzong.wang/mysql/data:/var/lib/mysql \ -p 3306:3306 \ - e MYSQL_ROOT_PASSWORD = 123456 \ mysql: 5.7# it argument: the container's Shell maps to the current Shell, and the commands you type in the native window are passed to the container.
After the container is started, it runs in the background.
Delete container files automatically after rm stops running.
# --name minei.net-mysql: minei.net-mysql
# - v: the directory (/ data/fuzong. Wang/mysql/data) is mapped to a container of/var/lib/mysql. Therefore, any changes to the specified directory are reflected in the container and accessed externally.
# -p maps the container's port 3306 to the host's port 3306.
-e Set the initial password to 123456
Use docker CP to copy the SQL file (/data/ fuzon.wang /db.sql) into the container.
Import the SQL file from the container using the source command.
Copy the codeAt this point, you can access mysql service resources through the IP of your new machine.
The individual parameters are explained here and not commented on later.
Generating a PHP container
Make PHP image
Since mysql extension of PHP is needed, the default PHP image cannot meet the requirements. You need to make image by yourself and add corresponding extension.
Create a new Dockerfile and write the following:
FROM php:7.2-fpm
RUN apt-get update \
&& docker-php-ext-install pdo_mysql mysqli \
&& docker-php-ext-enable pdo_mysql mysqli
# Two main things:
PHP 7.2;
# 2. Introduce mysql extensions
You can use the docker image build command to create an image fileDocker image build-t php7.2-with-extension:0.0.1Copy the codeGenerated container
Once you have the PHP image with the mysql extension ready, create the PHP container and run it:
Docker run -- the name PHP. The host - rm \ \ - d - v/var/WWW: / usr/share/nginx/HTML \ -p 9000:9000 \ php7.2 - with - the extension: 0.0.1Copy the codeNormally, the PHP container is already up and running.
Generate the NodeJS container
Vue SSR requires a Node running environment, so a program running environment is built on the new machine by Dorcker:
# run dockerdocker run --name minei.net-node.ssr \ -it -d \ --rm -p 8088:8088 \ -v /var/www/mimei.net.cn/ssr:/home/node \ Node: 10.10.0# - v directory (/ var/www/mimei.net.cn/ssr) deposit is hosting the vue code position
Copy the codeGenerate the Nginx container and configure it
Nginx container is relatively frequent changes, he has a lot of configuration files to deal with, need to map the regular conf file to the host, also need to install log files, SSL key files, etc., are mapped out separately, convenient for later changes and view.
Create a container
docker run --name mimei.net-nginx \
-d \
-p 80:80 \
-p 443:443 \
--link php.host:php.host \
--link minei.net-node.ssr:minei.net-node.ssr \
--privileged=true \
-v /var/www:/usr/share/nginx/html \ # site file directory, mapped to host
-v /opt/webservice/nginx/log:/var/log/nginx \ # log file
-v /opt/webservice/nginx/nginx.conf:/etc/nginx/nginx.conf \ The default configuration file
-v /opt/webservice/nginx/conf.d:/etc/nginx/conf.d \ # nginx vhost file
-v /opt/webservice/nginx/cert:/etc/nginx/cert \ # used to store HTTPSNginx: 1.16Copy the codeThis command has two more arguments than the previous one –link, which indicates that the nginx container is connected to PHP and nodeJS containers, and the colon, which indicates the alias of the container, can be accessed directly by others in the current container.
Configure nginx to forward requests to the appropriate service (nodejs, PHP)
# / opt/webservice/nginx/nginx. Conf here is a line
include /etc/nginx/conf.d/*.conf; # nginx will load all.conf files in the conf.d directory
Copy the codeConfiguration – Handles PHP requests
# so we create the corresponding file in the conf.d directory
# api.mimei.net.cn_https.conf server { listen 443 ssl http2; server_name api.mimei.net.cn; .# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
location ~ .php$ {
fastcgi_pass php.host:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; }... }Most of the configuration is normal nginx configuration
# fastcgi_pass php.host:9000; The PHP request is forwarded to the PHP container for processing.
Copy the codeConfiguration – Handling front-end page Requests (VUE SSR)
# 在 conf.d 目录下创建 ssr.mimei.net.cn_https.confserver { listen 443 ssl; server_name ssr.mimei.net.cn; . location / { proxy_pass http://minei.net-node.ssr:8088; }... }# proxy_pass http://minei.net-node.ssr:8088; Forward the request to the NodeJS container
Copy the codesummary
The main purpose of nginx is to forward requests, API requests to the PHP container, page requests to the NodeJS container, and finally put together a working website.
There are two points to note:
- The nginx port must be mapped to the host, I forgot to do it in time, always can’t find the service…
- The WWW directory of the PHP container must match the nginx mapping, or the target file may not be found.
Generate Jenkins container and initialize data
When the infrastructure is in place, Jenkins is actually going to be our main battle.
Generate Jenkins container
First, download Jenkins Image
docker pull jenkins/jenkins:latest
docker inspect IMAGE_ID Docker inspect retrieves metadata for containers/images
"ExposedPorts": {
"50000/tcp": {},
"8080/tcp": {}},..."Volumes": {
"/var/jenkins_home": {}}# You can see Jenkins' port, working directory, etc
# Prepare to run Jenkins later
Copy the codeLet’s get the Jinkins container up and running.
After docker finishes running (deleting Containers), related running data is also deleted. Therefore, data must be saved on the host to prevent loss.
mkdir /data/jenkins_home Create a working directory
docker run -d \
--name jenkins \
-p 8081:8080 \
-p 50000:50000 \
-v /data/jenkins_home:/var/jenkins_home \
jenkins/jenkins:latest
# -v data volume mount mapping (/data/jenkins_home: host directory, another container directory)
# Jenkins :2.60.3 Jenkins Mirror (current version)
Copy the codeUnder normal circumstances, Jenkins should be up and running.
But ~
Life is full of surprises
Check the running status of the ContainerDocker container PS -a Container ID IMAGE COMMAND CREATED STATUS PORTS NAMES C9D351DA530E Jenkins :2.60.3"/ bin/observatory - / usr/l..." 4 minutes ago Exited (1) 4 minutes ago jenkins
# Jenkins ran and quit
Take a closer look at the log
docker logs jenkins
touch: cannot touch '/var/jenkins_home/copy_reference_file.log': Permission denied
Can not write to /var/jenkins_home/copy_reference_file.log. Wrong volume permissions?
# Permissions
Change directory permissions
The Jenkins user uid in the container is 1000
chown -R 1000:1000 /data/jenkins_home/
Copy the codeRe-docker start CONTAINER_ID, and then you can see from docker ps that Jenkins is running.
Initialize data
Since I have mapped Jenkins’ merchandise to port 8081 on the host, I need to open this port in the admin background of the cloud server
After the container is running, Jenkins can be accessed through the IP address, and basically the following interface can be seen.
Follow the instructions, write the default data, and Jenkins will be done with the initialization.
Note the/var/jenkins_home/secrets/initialAdminPassword path inside the container, we’ve already cut down our path is mapped to the host machine, the actual view the address is:
cat /data/jenkins_home/secrets/initialAdminPassword
PASSWORD_STRING
Copy the codeUse Docker-compose to manage multiple containers
Although Docker simplifies the configuration of various environments, for this project, all these containers need to be managed and have different configurations. After a long time, some specific contents may be lost, and in fact, it is quite troublesome to manage.
Docker-compose provides us with a convenient way to manage the configuration of related containers into a docker-comemage. yml file, write the call relationship between several containers, and then with a single command, can start/shut down all containers at the same time.
Take our project as an example, write all the previous configuration information into the configuration file in the form of.yml.
Create a docker-comemess. yml file and write:
# configuration document at https://docs.docker.com/compose/compose-file/compose-file-v3/
version: "3"Services: mysql: image: mysql:5.7 container_name: minei.net-mysql networks: -net Privileged: PrivilegedtrueVolumes: - / data/fuzong. Wang/mysql/data: / var/lib/mysql ports: - 3306:3306 nodejs: image: node: 10.10.0 container_name: minei.net-node.ssr networks: - net stdin_open:true
tty: true
volumes:
- /var/www/mimei.net.cn/ssr:/home/node
Docker container exec it [container id] /bin/bash
CD /home/node&& node. /server.jsPHP: image: php7.2-with-extension:0.0.1 Container_name: PHP. Host Networks: -net volumes: - / var/WWW: / usr/share/nginx/HTML ports: - 9000:9000 nginx: image: nginx: 1.16 container_name: mimei.net - nginx networks: - net# external_links:
# - php.host:php-process
# - minei.net-node.ssr:node-server
volumes:
- /var/www:/usr/share/nginx/html
- /opt/webservice/nginx/log:/var/log/nginx
- /opt/webservice/nginx/nginx.conf:/etc/nginx/nginx.conf
- /opt/webservice/nginx/conf.d:/etc/nginx/conf.d
- /opt/webservice/nginx/cert:/etc/nginx/cert
ports:
- 80:80
- 443:443
jenkins:
image: jenkins/jenkins:latest
container_name: jenkins
networks:
- net
user: root
volumes:
- /data/jenkins_home:/var/jenkins_home
- /var/www:/var/www
ports:
- 8081:8080
- 50000:50000
networks:
net:
driver: bridge
Copy the codeThe content in docker-comemage. yml is basically the same as the previous command to create the container, except that the command has been rewritten in YML format.
The only difference is that when you create containers on the command line, you use the –link directive to indicate that container A is connected to container B. Networks are used here to place different containers on the same network so that containers can be accessed by other containers in the network.
Once the configuration file is written, we can start and stop all tasks with a single command.
Start all services
$ docker-compose up -d
Shut down all services
$ docker-compose stop
Copy the codeConfigure Jenkins to automate project release
With all the code, data, containers, and infrastructure in place, Jenkins is the final step to connect the dots.
Create an API build task
One extra thing to do before you start is that because the PHP project is based on the thinkPHP framework, the base library is not in a Git repository and needs to be manually copied from the old machine to the new one
Go to the directory where the API source code residesSCP - r ThinkPHP/[email protected]:/var/www/mimei.net/apiCopy the codeThen configure the build task
Create a free-style task named mimei-API
Source management Select Git, fill in the repository address, and add credential information.
Type: SSH Username with private key. Enter the Username and password when you create the git user.
Building a shell for the API is relatively simple; you simply copy the source code into the web service’s directory and make it accessible to the PHP processor.
How can different containers (Jenkins, nginx, PHP) directly access the same directory (/var/ WWW)? Docker-comemess. yml: docker-comemess. yml: docker-comemess. yml: docker-comemess. yml: docker-comemess. yml: docker-comemess. yml: docker-comemess. yml: docker-comemess. yml: docker-comemess. yml: docker-comemess. yml: docker-comemess. yml: docker-comemess. yml: docker-comemess. yml
php:
...
volumes:
- /var/www:/usr/share/nginx/html
nginx:
...
volumes:
- /var/www:/usr/share/nginx/html
jenkins:
...
volumes:
- /var/www:/var/www
Copy the codeAfter the configuration above, you should be able to build the project correctly. Here are some small test changes to the project:
$$ vim README.md
$$ git add .
$$ git commit -m 'add test string'
[master dc0c59c] add testString 1 file changed, 2 insertions(+), 1 deletion(-) $$git push [email protected]'s password: enumerate object: 5, done. Object count: 100% (5/5), done. Use 4 threads to compress the compressed object: 100% (3/3), complete. Write objects: 100% (3/3), 320 - byte | KiB/s, 320.00. A total of 3 differences (2), multiplexing 0 (0) difference To 82.156.7.129:/opt/git/mimei.net/mimei.api.git 0 b6558b.. dc0c59c master -> masterCopy the codeClick “Build Now” :
You can see that the code you just committed has been updated.
Since updates to the PHP code do not require recompilation or service restart, the API building process is complete.
Create a VUE SSR build task
The RELEASE of the Vue project is similar to the PHP project, with three differences in terms of the project page:
- Source code is hosted on Github (PHP source code is hosted on a private Git server)
- The code needs to be recompiled on the server side
- Major Restart the NodeJS service
Let’s look at the build task creation process in detail
First, build a free-style software project called mimei.SSR
Source control fills in the github address of the project, then adds credentials and specifies the branch as master
Build to copy the source code to the web service’s directory.
At this point, basically the same as the last project, the code is pulled, and then copied to the target directory; This project is different in that it needs to compile on the server side and then restart the service.
Since it is not possible to directly send instructions to nodeJS containers in Jenkins containers, the solution is to send instructions to NodeJS containers through remote shell execution.
Save the current configuration and return to Jenkins home page
Jenkins home -> System Administration -> System Configuration -> SSH Remote hosts
Add the host and corresponding credentials
The configuration page of the build task is displayed
Select Execute shell script on remote host using SSH and write the following shell.
ls -l ~
docker exec -i minei.net-node.ssr /bin/bash \ Minei.net -node. SSR is the name of the nodejs container
-c \ # Pass instructions into the container
"\ CD /home/node \ &&npm install -g pm2 \ # I am using pm2 to manage nodeJS program &&pm2 stop all \ # Stop all applications &&ls -l \ &npm Install dependencies && NPM run build # Build && NPM run start:pm2
Copy the codeOnce this is done, save and build, and if everything goes well you should see the following build information:
.
Because of the amount of build information being output, I’ve only taken a small screen shot here.
Once this is done, you can browse the website at https://ssr.mimei.net.cn
conclusion
This is a summary of the article, not to a traditional sense of ‘tutorial’, it is more a record of my process and ideas in configuring the front-end project release system. In the process of practice, in fact, a lot of problems, many of the pit, grope while solving, eventually form this article, however, the process of solving the problem of, most have not reflected in the article, so if you are in the process of practice, sure will encounter this or that kind of problem, don’t be afraid of, basic search engine can help we fix.
Finally, here are a few documents that have helped me a lot:
- Docker tutorial
- Docker Microservices tutorial
- Set up Git on the server
- Docker Compose configuration document: Compose file Version 3 Reference