This article is about the docker-related knowledge I learned and the pits I stepped on when DEPLOYING my personal blog, which is recorded and summarized here.
I. Introduction to Docker
A virtual machine is a solution with an ambient installation that runs one operating system inside another. Docker is a lightweight container engine running on Linux servers. Compared with traditional virtual machines, the biggest characteristic of Docker is that the container itself consumes few extra resources.
1. Main uses of Docker:
- Provide a disposable environment. For example, testing other people’s software locally, and providing unit testing and build environments for continuous integration.
- Provide resilient cloud services. Because Docker containers can be opened and closed at any time, it is very suitable for dynamic expansion and shrinkage.
- Build the microservice architecture. With multiple containers, a single machine can run multiple services, so a microservice architecture can be modeled locally.
2. Comparison between Docker and VIRTUAL machine:
3. Basic concepts of Docker
- Image
- Container
- Repository
- Host: the server that runs docker
Image: Docker packages the application and its dependencies in an image file. Can be a simple analogy for the computer installed system with the system disk, including the operating system, and the necessary software. For example, an image can contain a full centos operating system environment with Nginx and Tomcat servers installed. Note that the mirror is read-only. This makes sense, just as the system disks we burn are actually readable. We can use Docker images to view the list of local images.
Container: can be simply understood as providing the hardware environment of the system, it is the thing that really runs the project, consumes machine resources, and provides services. For example, we can temporarily think of the container as a Linux computer that runs directly. So, containers are started based on images, and each container is isolated from the other. Note that the container at startup creates a writable layer based on the image as the top layer. We can use Docker ps-a to view locally run containers.
Warehouse: Used to store images. This is very similar to Git. We can download images from the central repository or from our own repository. In the meantime, we can commit our image locally and then push it to a remote repository. Warehouses are divided into public warehouses and private warehouses. The largest public warehouse is the official warehouse Dock Hub. There are also many choices for public warehouses in China, such as Aliyun.
Reference materials:
- Docker Tutorial (Important)
- Docker from Beginning to Practice (important)
- The docker hub’s official website
- The front-end learn Docker
- Docker installation and easy to use
- Docker builds your first Node project to the server
Ii. Docker-related development process
The above figure shows the basic process of project development. We release a project according to the above process, taking my blog project (H5 project + Node service project) as an example.
We started with a working H5 project and a Node project (you can run the project from Node app.js), which we have now uploaded to Github.
Next, we need to install Docker on the server, start a Jenkins service through Docker, Jenkins pull code through Git, execute docker command through shell script, make image, and then start the service of Node project.
Docker installationreference
1. Install Docker
Lvm2 $sudo yum install -y yum-utils device-mapper-persistent-data lvm2 Here in ali cloud, for example $sudo yum - config - manager - add - repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo 3, update, yum $sudo yum makecache fast $sudo yum -y install docker-ce $sudo yum -y install docker-ce $sudo yum -y install docker-ce Accelerate the address access: https://cr.console.aliyun.com/cn-hangzhou/instances/mirrors?accounttraceid=29a4aadf7b7a4b2c89236b3e7b2f67efgbzb $ sudo mkdir -p /etc/docker $ sudo tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://tdcq9qtx.mirror.aliyuncs.com"} EOF $sudo systemctl daemon-reload $sudo systemctl restart docker // restart docker Login // Enter your docker_hub addressCopy the code2. Uninstall Docker
1, install a docker related query package yum list installed | grep docker containerd. IO. X86_64 1.2.6-3.3 el7 @ docker, ce, and stable X86_64 3:19.03.1-3. el7@docker-ce-stable docker-ce-stable cli.x86_64 1:19.03.1-3. el7@docker-ce-stable $sudo yum remove docker \ docker-client \ docker-client-latest \ docker-common \ docker-latest \ $sudo yum remove docker \ docker-client \ docker-client-latest \ docker-latest \ docker-latest-logrotate \ docker-logrotate \ docker-selinux \ docker-engine-selinux \ docker-engine $ rm -rf The/etc/systemd/system/docker. Service. D $rm - rf/var/lib/docker $rm - rf/var/run/docker to query the installation package, no information said uninstall cleanCopy the codeDocker common commands
Service docker start // Start the docker service systemctl start docker // Start the docker background service systemctl daemon-reload // Restart the docker daemon Systemctl restart docker // restart docker service docker pull Jenkins/Jenkins // docker pull image docker images // View the image list docker ps-a// View the container without adding-aView what is running, plus-aDocker rmI REPOSITORY docker rmI REPOSITORY docker rmI REPOSITORY docker rmI REPOSITORY Docker rmI button-API /v2 docker stop Container ID/ container alias // Close a started container docker start container ID/ container alias // start a closed container docker inspect container ID/ container alias // View details of a container dockerexec-it Container ID/ Container alias /bin/bash // Enter the container docker build --rm --no-cache=true-t jianghu-server. // Generate a mirror docker run-d--restart always --name jianghu-server -p 3006:3006 -v /var/jenkins_home/workspace/jianghu-server:/home/project Jianghu - Server // Run the containerCopy the code4, Docker install Jenkins
Reference: The use of Docker version Jenkins, using Docker + Jenkins automatic build, Docker container startup port mapping failure, Docker in Docker
1. Download Jenkins image
Find Jenkins image source from docker_hub, and then find Jenkins image source from docker_hub
$ docker pull jenkins/jenkins
Copy the code2. Start Jenkins container
$mkdir /var/jenkins_home $sudo chown 777 /var/jenkins_home $docker run-d -p 8080:8080 -p 50000:50000 -v /var/jenkins_home:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker -v /etc/localtime:/etc/localtime --name jenkins docker.io/jenkins/jenkins
-d-p 8080:8080 Run the image in the background. Map port 8080 of the image to port 8080 of the server. The JNLP-based Jenkins agent communicates with the Jenkins Master through TCP port 50000, so it also maps -v /var/jenkins_home: The /var/jenkins_home directory is the Jenkins working directory. We mounted a directory on the hard disk to this location so that we could continue to use the original working directory after updating the image. - v/var/run/docker. The sock: / var/run/docker. The sock Jenkins container docker are needed to perform the task, So here will docker command also mapped to container - v/usr/bin/docker: / usr/bin/docker will docker commands mapped to a container - v/etc/localtime: / etc/localtime Let the container use the same time Settings as the server. - the name myjenkins to name the container an alias docker. IO/Jenkins Jenkins using just download a mirror image of the installation errors that may occur: 1."docker run"The requires the at further argument 1. Reason: command less parameters, namely command Jenkins installation need to face with the source: docker. IO/Jenkins Jenkins 2, the Error response from the daemon: driver failed programming external connectivity on endpoint jenkins (db9d8092ddb37d4243647cad8c63555e090b9 161d1f3c90db8d8c43281e9c86e): (iptables failed: iptables --wait -t nat -A DOCKER -p tcp-d0/0 --dport 8081 -j DNAT --to-destination 172.17.0.2:8080! -i docker0:iptables: No chain/target/match by that name. The reason: Port mapping fails. $sudo systemctl restart Docker https://cao0507.github.io/2018/12/05/docker%E5%AE%B9%E5%99%A8%E5%90%AF%E5%8A%A8%E6%97%B6%E7%AB%AF%E5%8F%A3%E6%98%A0%E5%B 0% 84% B4 E8 B1 A4 E5% % % % % A5/3, the docker:commandNot found Jenkins There is no docker command in the container, so the docker command can be found inside the container by hanging in the docker command directory, and the host docker can execute related commands by mounting docker.sockCopy the code3. Jenkins basic configuration
1. Initialize Jenkins
Enter Jenkins’ initial login password at http://120.XXXX:8081/ and run the following command
$ docker exec jenkins tail /var/jenkins_home/secrets/initialAdminPassword
76f4ac68586b4e14bb0f7f9c43912e92
Copy the codeThen copy the password to log in and install the recommended plug-in. After installing it, set Jenkins login account password
2. Configure Jenkins SSH
SSH here is configured for the container, allowing Jenkins to pull the code from github
1. Enter the internal docker of container Jenkinsexec-it Jenkins /bin/bash 2. Generate SSH key file ssh-keygen -t rsa-c"[email protected]"3. Add the contents of /var/jenkins_home/.ssh/id_rsa.pub to the.ssh/authorized_keys file of the host, or run the following command to manually copy the contents:exit && cd~ // Exit the container mkdir.ssh &&cdSsh/id_rsa. pub: /var/jenkins_home/.ssh/id_rsa.pub: /var/jenkins_home/.ssh/id_rsa.pub: /var/jenkins_home/.ssh/id_rsa.pub: /var/jenkins_home/.ssh/id_rsa.pub: /var/jenkins_home/.echo "`tail /var/jenkins_home/.ssh/id_rsa.pub`">> authorized_keys :cat /var/jenkins_home/.ssh/id_rsa.pub >> authorized_keys: https://www.jianshu.com/p/e34674f34242 SSH - copy - id - I ~ /. SSH/id_rsa pub < username > @ < host > 4, restart the SSH service systemctl restart Sshd. service 5. Configure the public key in your GIthub/Gitlab SSH SettingCopy the codeJenkins Publish over SSH
1. Install Publish Over SSH
Select Publish Over SSH, and click Install directly.
If the installation fails, go to the plug-in download address, and then go to plug-in management – Advanced – upload the plug-in
2. Configure SSH
System Administration > System Settings > Publish over SSH
This is where the project is packaged and compiled and sent to which server and which directory
4. Configure the Node plug-in
Since our Node project requires node compilation and packaging, install the Node plug-in system configuration – Plug-in Management – Optional plug-ins, search nodeJS, and install it directly
Then go to System Configuration – Global Tools Management -nodejs and configure the versions of Node you want to use
Jenkins launches Web project
The Web project uses Nginx to access the Web project, so install nginx first.
1, Docker install nginx
1. Download the image
docker pull nginx
Copy the code2. Start the nginx container
$ docker run -d-p 80:80 --name nginx nginx if you use the public IP address of your server, you can view the nginx welcome pageCopy the code3. Mount the configuration file
Since the Nginx configuration file in step 2 is inside the container, it is not easy to modify, so we can first copy the configuration file inside the container to the host
Nginx/WWW nginx/logs copy the configuration file to the working directory $docker cp-a nginx:/etc/nginx/nginx.conf ~/nginx/conf
Copy the code4. Restart the nginx container
Stop nginx docker rm nginx restart the container and mount the working directory docker run-d --restart always -p 80:80 -v ~/nginx/www:/usr/share/nginx/html -v ~/nginx/conf/nginx.conf:/etc/nginx/nginx.conf -v ~/nginx/logs:/var/log/ nginx - name nginx nginx, restart always said / / docker restart automatically restart the container - v ~ / nginx/WWW: / usr/share/nginx/HTML / / mapping the default static page container path - v ~ / nginx/conf/nginx. Conf: / etc/nginx/nginx. Conf / / map container configuration file - v ~ / nginx/logs: / var /log/nginx // Map the container log directoryCopy the codeGo to your server’s IP address and find 403, because the WWW directory has no content, go to create a new index.html
$ cd~/nginx/ WWW $touch index.html DOCTYPE html> <html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="Width = device - width, initial - scale = 1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge"> <title>Document</title> </head> <body> </h1> </body> </ HTML > find that you can see this page againCopy the code2. Create a Web project deployment taskreference
1. Create a task – fill in the task name – build a free-style software project
2. Source code management
SSH has been configured in step 4. You can use the GIT project address of SSH protocol
[root@iZgm2xqy8dx52vZ ~]# docker exec -it jenkins /bin/bashjenkins@8f133f801f50:/$git --version // jenkins@8f133f801f50:/$represents v2.11.0 in the Jenkins containerCopy the codeIf not, go to System Configuration – Global Tools Configuration -git and check automatic installation
If the Git version is too old, try installing/uninstalling/reinstalling Git in Jenkins below.
If the git address of the container is correct and SSH has been configured, the project git address is changed to HTTPS. If there is no problem, the project git address is switched back to SSH. If there are more questions!!
As the happiest man in the league, I’m not going to give up. Add the private keys to the Credentials and click Add
And then, yo-ho, finally!
3. Build the environment
Select the Node plug-in you configured
4, build,
Web project, remove dependencies – reinstall dependencies – package compile – compress
5. Operation after construction
Package and compile the package and send it to the service to be deployed. The final transmission path is Remote directory of SSH server + Remote directory here. For example, I configured the root directory/earlier, and here is /home/umipc, so the final transmission path is: To view the /home/umipc of the SSH server you want to deploy, go to the directory of that server.
6. Perform the build operation
Click Build now and wait for a while. If nothing goes wrong, the project builds successfully. Build details We can look at the Console Output of this build to see how the project is executing. Here the build may fail and several common errors are recorded:
If the file permission is insufficient, the solution is as follows: chmod -r 777 /var/jenkins_home/workspace 2. If the Node version is inappropriate, switch to another versionCopy the code
ERROR: Timeout after 10 minutes ERROR: ERROR cloning remote repo'origin'You can set the timeout and pull code configuration as followsCopy the code
7, modify the nginx configuration agent project
$ cd~/nginx/conf $vim nginx.conf nginx.conf worker_processes 4; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; server { listen 80; server_name localhost; location / { root /usr/share/nginx/html/umiMobile; index index.html index.htm; try_files$uri $uri/ /index.html;
}
location ^~ /static/ {
root html;
}
Proxy server interfaceThe location/API / {proxy_pass http://172.16.187.16:3006/;Proxy interface address}}} root/usr/share/nginx/HTML/umiMobile deposit for the container you in Jenkins static path of the pageCopy the codeChanges to nginx.conf will take effect only after the nginx service is restarted
Docker restart nginx // Restart the nginx serviceCopy the codeIf nothing else, revisit the IP and you’ll see your own project page. Quack
If the container does not start properly, go to the ~/nginx/logs directory and check the error logs. You will be pleasantly surprised.
3. Git installation in Jenkins
Remember: the following operations are done in Jenkins containers!!
1. Check Git information
First go into the container to see if Git is installed
Enter the Jenkins container $dockerexec-it Jenkins /bin/bash $git --version it version 1.7.1 fastestmirror Loading mirror speeds from cached hostfile Installed Packages Name : git Arch : x86_64 Version : 1.8.3.1 Release: 20.el7 Size: 22 M Repo: installed From Repo: updates Summary: Fast Version Control System URL: http://git-scm.com/ License : GPLv2 Description : Git is a fast, scalable, distributed revision control system with anCopy the codeThe highest git installed on yum is up to 1.8, so you need to compile and install it yourself by pulling git files
2. Install the dependency libraries
$ cd$yum install curl-devel expat-devel gettext-devel openssl-devel zlib-devel $yum install GCC $yum install -devel openssl-devel zlib-devel $yum install GCC perl-ExtUtils-MakeMakerCopy the code3, Uninstall the older version of Git, and then uninstall it again because the default installation dependency is git installed!!
# yum remove git
Copy the code4. Download a later version of GitDownload address view
$$tar wget https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.11.0.tar.gz - ZXVF git - 2.11.0. Tar. Gz / / $cdGit - 2.11.0Copy the code5. Compile and install
# make prefix=/usr/local/git all
# make prefix=/usr/local/git install
Copy the codeAdd git’s Path to the environment variable
# echo "export PATH=$PATH:/usr/local/git/bin" >> /etc/bashrc
# source /etc/bashrc // enable the environment configuration file
Git --version // check whether the installation was successful
Copy the codePs: The git in the container is not the same as the Git on the host. Keep in mind that the container and the host are isolated.
Deploy the Node project through Jenkins + Docker
Here we will create an image for the Node project and use this image to start the Node service. There are several ways to create an image. Here we use Dokerfile.
1. Add the Docker image file
In your Reagan directory for the Node project, create a new file called Dockerfile that looks like this:
FROM the node: 12.13If the current directory does not exist, it will be created automatically
WORKDIR /usr/src/app
Copy the context files to the container working directory separately from the following to ensure that the image layer does not need to reinstall dependencies if the dependency files are not changed
COPY package*.json ./
Install project dependencies
RUN npm install --registry=https://registry.npm.taobao.org
# copy the context to the project file to the working directory
COPY . ./
Expose port 3006 of the container
EXPOSE 3006
Start the Node service
CMD [ "node"."app.js" ]
Copy the codeDocker: start the node service by using this image. Dockerignore: Start the node service by using this image.
node_modules
.eslintignore
.eslintrc.js
jest.config.js
README.md
Copy the codeThis file acts like.gitignore, which files to ignore when copying project files to the container.
2. Add a Node task to Jenkins
1. Create a process
Add the project git address as you did above to create a Web project
Build environment select Send Files or execute Commands over SSH after the build runs and configure as follows:
The Exec Command section reads as follows:
# Switch to the source directory corresponding to jenkins-home workspace
cd /var/jenkins_home/workspace/jianghu-server
pwd
# set the tag
# image_version=`date +%Y%m%d%H%M`;
# echo $image_version;
# Stop the Previous Docker container
docker stop jianghu-server;
Delete the container
docker rm jianghu-server;
Delete the container
docker rmi jianghu-server:latest;
# build image
docker build --rm --no-cache=true -t jianghu-server .;
docker images;
# Run the image you just built
docker run -d --restart always --name jianghu-server -p 3006:3006 -v /var/jenkins_home/workspace/jianghu-server:/home/project jianghu-server:latest;
docker logs jianghu-server;
Copy the codeThe command here is to create an image from the Node project Dockerfile, and then use the image to start the container and run the Node service. Save the application, hit Build Now, and if all goes well you’ll find that the service is already accessible.
If it doesn’t work out, check to see if it’s the problem below
2. Frequently asked Questions
1. If the task fails to execute, go to the console output of the task record to check the specific reason, which is generally no Docker command in the container or insufficient permissions. Check whether there is any omission in the steps of installing Jenkins. 2. If the task is successfully executed and the node service fails to be accessed, the node port or firewall port on your server may be closed. If the port is not opened on the server, open it in the network and security group of the server instance. You can run the following command to view the firewall port. CentOs7.0 began using the firewall as a firewall instead of iptables, and opened ports in a different way. Firewall-cmd --state // Check firewall status, Service // Starts the firewall systemctl stop firewalld.service // Starts the firewall systemctl restart Firewalld. service // Restart the firewall firewall-cmd --reload // reload the configuration firewall-cmd --query-port=8080/ TCP // Check whether the port is firewall-cmd --zone=public --list-ports firewall- CMD --zone=public --add-port=8080/ TCP --permanent Zone =public: Indicates that the scope is public. Add-port =8080/ TCP: adds TCP port 8080. Permanent: indicates that the service takes effect permanently. If this parameter is not set, the service only lasts for the current service life cycle and becomes invalid after the service is restarted. Firewall-cmd --permanent --remove-port=666/ TCP // Disable the portCopy the codeAfter everything is done, check that the service is running happily
Install Mysql
My Node service is connected to the mysql database, so I will install mysql as well
1. Download the image
Go to docker.hub and search for mysql images and download them
$docker pull mysql // Download the latest image $docker images-a/ / see all mirror $docker images | grep mysql / / check mysql mirror $mysql -- help | grep Distrib / / see the mysql versionCopy the code2. Start the mysql container
docker run --name mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=123456 -dMysql -p 3306:3306: map port 3306 of the container to port 3306 of the host -v$PWD/ conf/my. CNF: / etc/mysql/my CNF: will host the current directory of the conf/my CNF mounted to the container of the/etc/mysql/my CNF - v$PWD/logs:/logs: Mount the logs directory under the current directory of the host to /logs -v of the container$PWD/data:/mysql_data: mount the data directory in the current directory of the host to /mysql_data of the container-eMYSQL_ROOT_PASSWORD=123456: initialize root user password $docker ps check whether the mysql container is runningCopy the code3, log in to mysql and create a remote connectionreference
$ docker exec-it mysql bash $mysql -u root -p $CREATE USER'jianhao'@The '%' IDENTIFIED WITH mysql_native_password BY '123456'; TO set the remote login account password, run the command $GRANT ALL PRIVILEGES ON *.* TO'jianhao'@The '%'; // Enable remote access privileges $flush PRIVILEGES; // Refresh permissionsCopy the codeThen use Navicat connection to check, done!! The Node project can then connect to the database via IP :3306.
To be continued…
