How to migrate existing applications to Choerodon Toothfish (part 2)

• Application environment construction

• Database Migration

• Application migration

• Use the Choerodon toothfish

The following details how to migrate a general application to Choerodon and use Choerodon as a support platform for development, distribution, and deployment, using the example of migrating an asset cloud platform to a Choerodon toothfish.

Application system environment construction

Preparing the server

Install the Kubernetes cluster

The Kubernetes cluster is installed in a standard way, and Choerodon has compiled a detailed and convenient Kubernetes installation document for reference. There are three installation modes: local VM installation mode, server installation mode, and cloud installation mode. You can select the installation mode based on your own requirements.

Database Migration

If the original environment does not have data, skip this section.

If the new cluster can connect to your old database, you can choose to use the old database connection information without doing anything to the database. If the original database cannot be connected, back up key data such as database configuration and data.

Specific operations are as follows:

1. Determine the data to be backed up, such as configuration and data. The memory size can be adjusted based on the actual database.
2. Deploy the new database to the new cluster. You are advised to use the helm for deployment. For details, see mysql Deployment.
3. Use the tools provided with the database to restore data to the new database.
4. Check whether the database is running properly.

Application migration

Create a project

1. Log in to the system as an organization administrator
2. Click on the organization, such as “Hander R&D”, to enter the management menu of the organization layer

3. In the management menu of the organization layer, click “Hande R&D” -> menu in the upper left corner -> “Organization Settings” -> “Project Management”. The project management and Creation page is displayed.

4. On the Project Management page, click Create Project. In the dialog box that is displayed, enter the project number and project name.

For example,

Project code: Hand-RDC-HALM

Project name: Hande Asset Cloud Platform

5. Click Create to complete the creation. Once the project is created, users can use Choerodon’s system features such as knowledge management, agile management, development pipeline, test management, deployment pipeline, and more.

Create an

Apply containerized configuration

Choerodon: Using the containerization concept

The Helm is to Kubernetes what yum is to RHEL, or apt-get is to Ubuntu. Helm manages applications using Charts, which, like RPM, describes applications and their dependencies.

Therefore, it is important to include the Charts folder in Choerodon’s standard application code structure, as shown in the screenshot below, which is a standard structure for a back-end project.

• Templates is a template file, which is rendered into an actual file and sent to Kubernetes.

• Values. Yaml is a predefined variable of the template.

• Chart.yaml contains the Chart version information description, which you can access from the template.

• Deployment.yaml: Creates a basic listing for a Kubernetes deployment.

• Service.yaml: Create a basic listing of service endpoints for your deployment.

• Helpers.tpl: a place to put the template helper that you can reuse throughout the chart.

▌ Creating basic application images

What is application base mirroring? To start with, an application base image is usually pre-installed with the basic dependency tools and system configurations for building, testing, and running applications, such as utility classes, dependency packages, and system consistency Settings.

The asset cloud application is a PHP project, so the application base image should be the PHP running environment. First, go to DockerHub to search whether there is an official public image, which can be further customized to generate the required image, or it can be customized from a basic system-only image.

1. When writing Dockerfile custom base image, try to make the image size smaller and write the image layer as little as possible. Only add relevant components necessary for application running, do not add unnecessary things to enter. Create a file named dockerfile. base in the project root directory and write the information defined by Dockerfile to the file, for example:

 1# Use Ubuntu as the system2 the from ubuntu: 16.04 3Set environment variables
 4ENV NODE_HOME=/usr/local/node8
 5ENV PATH=$NODE_HOME/bin:$PATH
 6ENV COMPOSER_ALLOW_SUPERUSER=1
 7ENV COMPOSER_HOME=/composer
 8ENV USER=root
 9ENV SASS_BINARY_PATH=/opt/linux-x64-57_binding.node
10
11Add the source and install the required software
12RUN echo "deb-src http://archive.ubuntu.com/ubuntu xenial main restricted #Added by software-properties" > /etc/apt/sources.list \
13    && echo "deb http://mirrors.aliyun.com/ubuntu/ xenial main restricted" >> /etc/apt/sources.list \
14    && echo "deb-src http://mirrors.aliyun.com/ubuntu/ xenial main restricted multiverse universe #Added by software-properties" >> /etc/apt/sources.list \
15    && echo "deb http://mirrors.aliyun.com/ubuntu/ xenial-updates main restricted" >> /etc/apt/sources.list \
16    && echo "deb-src http://mirrors.aliyun.com/ubuntu/ xenial-updates main restricted multiverse universe #Added by software-properties" >> /etc/apt/sources.list \
17    && echo "deb http://mirrors.aliyun.com/ubuntu/ xenial universe" >> /etc/apt/sources.list \
18    && echo "deb http://mirrors.aliyun.com/ubuntu/ xenial-updates universe" >> /etc/apt/sources.list \
19    && echo "deb http://mirrors.aliyun.com/ubuntu/ xenial multiverse" >> /etc/apt/sources.list \
20    && echo "deb http://mirrors.aliyun.com/ubuntu/ xenial-updates multiverse" >> /etc/apt/sources.list \
21    && echo "deb http://mirrors.aliyun.com/ubuntu/ xenial-backports main restricted universe multiverse" >> /etc/apt/sources.list \
22    && echo "deb-src http://mirrors.aliyun.com/ubuntu/ xenial-backports main restricted universe multiverse #Added by software-properties" >> /etc/apt/sources.list \
23    && echo "deb http://archive.canonical.com/ubuntu xenial partner" >> /etc/apt/sources.list \
24    && echo "deb-src http://archive.canonical.com/ubuntu xenial partner" >> /etc/apt/sources.list \
25    && echo "deb http://mirrors.aliyun.com/ubuntu/ xenial-security main restricted" >> /etc/apt/sources.list \
26    && echo "deb-src http://mirrors.aliyun.com/ubuntu/ xenial-security main restricted multiverse universe #Added by software-properties" >> /etc/apt/sources.list \
27    && echo "deb http://mirrors.aliyun.com/ubuntu/ xenial-security universe" >> /etc/apt/sources.list \
28    && echo "deb http://mirrors.aliyun.com/ubuntu/ xenial-security multiverse">> /etc/apt/sources.list \ 29 30 && apt-get update \ 31 && apt-get install -y gcc g++ make xz-utils axel libsass-dev libmcrypt-dev curl supervisor rpm libaio1 libaio-dev \ 32 && apt-get install -y nginx \ 33 && apt-get install -y php-fpm  php-pdo-mysql php-pdo-sqlite php-curl php-redis php-mongodb \ 34 php-gd php-mcrypt php-mbstring php-xml php-ldap php-imap php-zip php-dom php-soap php-dev phpunit \ 35 36 && apt-get autoclean \ 37 && apt-get clean \ 38 && rm -rf /var/lib/apt/lists/* \ 39 &&echo "daemon off;" >> /etc/nginx/nginx.conf \
40    && sed -i "s/^user\swww-data; /user root; /g" /etc/nginx/nginx.conf \
41    && sed -i "s/^user\s=\swww-data/user = root/g"The/etc/PHP / 7.0 / FPM/pool. 42 && d/www.conf \ sed - I"s/^group\s=\swww-data/group = root/g"The/etc/PHP / 7.0 / FPM/pool. 43 && d/www.conf \ sed - I"s/^; daemonize\s*=\s*yes/daemonize = no/g"/etc/php.7.0/fpm /php-fpm.conf \ 44 && sed -i"s/^; mbstring\.internal_encoding\s*=.*$/mbstring\.internal_encoding = UTF-8/g"/etc/php.7.0/fpm /php.ini \ 45 && sed -i"s/^; mbstring\.internal_encoding\s*=.*$/mbstring\.internal_encoding = UTF-8/g"/ etc/PHP / 7.0 / cli/PHP ini \ 46 &&echo "[program:nginx]\ncommand=/usr/sbin/nginx" >> /etc/supervisor/conf.d/nginx.conf \
47    && echo "[program:php-fpm7.0]\ncommand=/usr/sbin/php-fpm7.0 --nodaemonize -- FPM -config /etc/php-7.0/fpm /php-fpm.conf -r" >> /etc/supervisor/conf.d/php.conf \
48    && mkdir /run/php \
49
50    && sed -i "s/\[supervisord\]/[supervisord]\nnodaemon=true\nuser=root\n/g"The/etc/supervisor/supervisord. Conf \ 51 52 && axel -n 20 https://getcomposer.org/download/1.4.2/composer.phar \ 53 && mv composer.phar /usr/local/bin/composer \
54    && chmod +x /usr/local/bin/composer \
55    && composer config -g repo.packagist composer https://packagist.phpcomposer.com \
56
57    && cd /usr/local58 && \ axel -n 10 https://nodejs.org/dist/v8.2.1/node-v8.2.1-linux-x64.tar.xz \ 59 && xz-dXz \ 60&& tar XVF node-v8.2.1-linux-x64.tar \ 61&& unlink /usr/local/node-v8.2.1-linux-x64.tar \ 62&&mv node* node8\63&&chown -r root:root node8\64&&npm install CNPM -g --registry=https://registry.npm.taobao.org \ 65 && npm install nrm -g -registry=https://registry.npm.taobao.org \ 66 && nrm use taobao 67# Exposed port
68EXPOSE 80 443
69Set the default startup command
70CMD ["/usr/bin/supervisord"."-c"."/etc/supervisor/supervisord.conf"]
Copy the code

2. In the project root directory command executed application base image building, built after push mirror image to the warehouse, such as registry.choerodon.com.cn/hand-rdc-halm warehouse (note: This repository will be created automatically when Choerodon creates a project:

1 docker build -t registry.choerodon.com.cn/hand-rdc-halm/php:ubuntu-16.04-f Dockerfile.base .
Copy the code

3. Push the constructed image to the image repository:

1 docker login registry.choerodon.com.cn 2 docker push -t registry.choerodon.com.cn/hand-rdc-halm/php:ubuntu-16.04Copy the code

Dockerfile and Helm Chart were added to the original codebase

▧ Dockerfile

It is easy to write an application Dockerfile on the basis of the application base image, just put the program into the base image specific directory, set up the image to run pre-processing and start the command.

In this case, the application runtime needs to link to the database, but the database information is only known by the runtime. The solution is to configure this information as environment variables, and the mirror runtime retrieves this information from the environment variables and replaces it with the corresponding configuration file.

Step 1: Modify the configuration file and write the startup script

Create the auto_devops folder in the root directory of the project and copy the configuration file to the folder. The following is the database configuration fragment in the configuration file. Replace the changed values with uppercase letters and ensure that they are unique in the file.

 1'dbconfig' =>
 2    array(
 3        'db_host_name'= >'DB_HOST_NAME'4,'db_host_instance'= >'SQLEXPRESS'May,'db_user_name'= >'DB_USER_NAME'6,'db_password'= >'DB_PASSWORD'7,'db_name'= >'DB_NAME'8,'db_type'= >'mysql'9,'db_port'= >'DB_PORT'10,'db_manager'= >'MysqliManager'11),Copy the code

Then create a new docker-entrypoint.sh file in the auto_devops folder and write a startup script that replaces the capitalized variable names in the configuration file.

1#! /bin/bash
2
3sed -i "s DB_HOST_NAME $DB_HOST_NAME g"  /var/www/config.php 
4sed -i "s DB_USER_NAME $DB_USER_NAME g"  /var/www/config.php 
5sed -i "s DB_PASSWORD $DB_PASSWORD g"  /var/www/config.php 
6sed -i "s DB_NAME $DB_NAME g"  /var/www/config.php 
7sed -i "s DB_PORT $DB_PORT g"  /var/www/config.php 
8
9exec "$@"Copy the code

Step 2: Write the application Dockerfile

Create a new file named Dockerfile in the project root directory

 1# Application base mirroring2 the from registry.choerodon.com.cn/hand-rdc-halm/php:ubuntu-16.04 3Copy the program to the /var/www/ directory
 4COPY . /var/www/
 5Copy the configuration file to the corresponding directory
 6COPY ./auto_devops/config.php    /var/www/config.php
 7Copy the script that replaces the environment variable to the image
 8COPY ./auto_devops/docker-entrypoint.sh      /docker-entrypoint.sh
 9This command is used to run the image by default
10ENTRYPOINT ["/bin/sh"."/docker-entrypoint.sh"11]# start service
12CMD ["/usr/bin/supervisord"."-c"."/etc/supervisor/supervisord.conf"]Copy the code

▧ Helm Chart

You need to understand the objects and concepts in Kubernetes before writing Helm Chart. In this case, you only need to define a Deployment object to run the application.

Step 1: Create a directory

Create the following directory structure in the root directory of the project, first create a folder named chart, then create a folder with the same name as the application, in this case Helm Chart, and then create a templates directory in the halm-dev folder.

1chart 2 ├── ├── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ──Copy the code

Step 2: Write the _helpers.tpl file

Define some common lables or values in the templates folder into the _helpers.tpl file:

1{{/* vim: set filetype=mustache: */}}
2{{- /*
3service.labels.standard prints the standard service Helm labels.
4The standard labels are frequently used in metadata.
5*/ -}}
6{{- define "service.labels.standard" -}}
7choerodon.io/release: {{ .Release.Name | quote }}
8{{- end -}}
Copy the code

Step 3: Write the deployment.yml file

In the Templates folder, create a file called deployment.yml with the following contents:

 1apiVersion: apps/v1beta2
 2kind: Deployment
 3metadata:
 4  name: {{ .Release.Name }}
 5  labels:
 6{{ include "service.labels.standard" . | indent 4 }}
 7spec:
 8  replicas: {{ .Values.replicaCount }}
 9  selector:
10    matchLabels:
11{{ include "service.labels.standard" . | indent 6 }}
12  template:
13    metadata:
14      labels:
15{{ include "service.labels.standard" . | indent 8 }}
16    spec:
17      containers:
18        - name: {{ .Release.Name }}
19          image: "{{ .Values.image.repository }}:{{ .Chart.Version }}"
20          imagePullPolicy: {{ .Values.image.pullPolicy }}
21          env:
22{{- range $name.$value := .Values.env }}
23{{- if not (empty $value) }}
24          - name: {{ $name | quote }}
25            value: {{ $value | quote }}
26{{- end }}
27{{- end }}
28          ports:
29            - name: http
30              containerPort: {{ .Values.service.port }}
31              protocol: TCP
32          resources:
33{{ toYaml .Values.resources | indent 12 }}
Copy the code

Step 4: Write the chart.yaml file

Write chart. yaml file in the halm-dev folder. This file contains information about the application.

 1# API version
 2apiVersion: v1
 3# App Version
 4appVersion: "1.0"
 5# Application description
 6description: A Helm chart for Kubernetes
 7# app name
 8name: halm-dev
 9# Apply chart versionVersion: 0.1.0 fromCopy the code

Step 5: Write the document

Write values. Yaml in the halm-dev folder, which contains the variables and default values that will be used in the deployment.yml file in the templates folder.

 1# Declare variables to be passed into your templates.
 2
 3replicaCount: 1
 4
 5image:
 6  repository: registry.choerodon.com.cn/hand-rdc-halm/halm-dev
 7  pullPolicy: Always
 8
 9env:
10  SITE_URL: http://localhost:8081/
11  DB_HOST_NAME: 
12  DB_USER_NAME: 
13  DB_PASSWORD: 
14  DB_NAME: 
15  DB_PORT: "3306"
16
17logs:
18  parser: nginx
19
20resources: 
21  # We usually recommend not to specify default resources and to leave this as a conscious
22  # choice for the user. This also increases chances charts run on environments with little
23  # resources,such as Minikube. If you do want to specify resources,uncomment the following
24  # lines,adjust them as necessary,and remove the curly braces after 'resources:'.
25  limits:
26    # cpu: 100m
27    # memory: 2Gi
28  requests:
29    # cpu: 100m
30    # memory: 1Gi
Copy the code

For more details on how to Charts, see the Documentation on the Choerodon website from YAML to Helm.

CI continuous integration configuration

In the previous section “Application Container Configuration,” it was mentioned that the Choerodon standard application source code structure must include the Charts file. Also, Choerodon uses Gitlab-ci as a CI tool, so you need to add a.gitlab-ci.yml file to the application source code.

The main work in CI is to build image and generate Chart package, and finally upload Chart package to Choerodon for integration with Choerodon.

Create a. Gitlab-ci. yml file in the root directory of the project and paste the following contents:

 1Set the CI runtime environment imageImage 2: registry.cn-hangzhou.aliyuncs.com/choerodon-tools/cibase:0.6.0, 3, 4# setup phase, where only image building and Chart package generation are carried out, so define it as a phase
 5stages:
 6- docker-build
 7
 8docker-build:
 9  stage: docker-build
10  Commands that need to be executed in the phase
11  script:
12  - docker_build
13  - chart_build
14
15A global script is defined. Before each stage is run, the following code is executed to retrieve the corresponding variables and encapsulated shell functions from the Choerodon platform.
16.auto_devops: &auto_devops |
17    http_status_code=`curl -o .auto_devops.sh -s -m 10 --connect-timeout 10 -w %{http_code} "${CHOERODON_URL}/devops/ci? token=${Token}"18 `if [ "$http_status_code"! ="200" ]; then
19      cat .auto_devops.sh
20      exit1 the 21stfi
22    source .auto_devops.sh
23    # rewrite the docker_build function
24    function docker_build(){
25        docker build --pull -t ${DOCKER_REGISTRY}/${GROUP_NAME}/${PROJECT_NAME}:${CI_COMMIT_TAG} .
26        docker push ${DOCKER_REGISTRY}/${GROUP_NAME}/${PROJECT_NAME}:${CI_COMMIT_TAG}
27    }
28
29before_script:
30  - *auto_devops
Copy the code

For more information on how to configure to comply with Choerodon standards and requirements, please refer to the. Gitlab-ci. yML file on the Choerodon website.

Replace the original code base with the Choerodon code base

Build a new version

Create an environment

Deploy the newly generated version

The deployable version is ready, the environment is ready, and now you need to deploy the deployable version into the environment.

1. Go to the Choerodon application deployment page, system path: “Hande R&D” (organization)-> “< your Project >” -> Deployment Pipeline -> Application Deployment.

2. Select the application and version.

For example,

• Application selected: Asset Cloud Application (Halm-dev)

Select version: 2018.8.27-234043-master

3. Select an environment and modify the configuration information.

For example,

• Select environment: Development environment

Also, the following configuration information can be modified to suit your needs.

4. Select a deployment mode.

For example,

• Select a deployment mode: Create an instance

For the first deployment, select New Instance.

5. Confirm information and deployment.

6. After verifying the information, deploy it. You can view the deployment status on the Instances page. The final instance name is halm-dev-9fc8.

Create a network

After applications are deployed, they cannot be accessed by external users. You need to create networks and domain names. Now create the network.

1. Go to the Choerodon Web page, system path: “Hander R&D” (organization)-> “< your project >” -> Deployment Pipeline -> Network.

2. Click Create Network. The Create network page is displayed. Fill in the relevant field information.

For example,

• Environment: Select Development Environment.

• Target object: Select Select Instance.

• Application Name: Select Asset Cloud Application.

• Select an instance: Select halm-dev-9fc8, which is the instance generated in the previous deployment.

• Network Configuration: Select ClusterIP

• External IP: NULL

• Port: 80, port of the internal application of the mirror

• Destination port: 80, used by K8s applications to provide services

• Network name: halm-dev-3491

Create a domain name

Have a network but also have a domain name.

1. Go to the Choerodon domain name page, system path: “Hander R&D” (organization)-> “< your project >” -> Deployment Pipeline -> Domain name.

2. Click Create Domain name. On the page that is displayed, enter related information.

Test access

Knowledge management

Knowledge management mainly provides the functions of Wiki. Users can use knowledge management to do project approval, requirements analysis, application design, etc., and also record Scrum review meetings. In short, it can be used as a media for daily document editing and sharing of projects.

Information:

Quick start on knowledge management

Knowledge Management User manual

[Official website video]

[Video on Knowledge Management]

Agile management

Agile management mainly provides project management functions, including requirements management (stories, user story maps, issues), planning (backlog, sprints), execution (activity sprints, kanban), and data analysis related to project management, such as cumulative flow charts, burnout charts, etc.

Information:

Quick Start on Agile Management

Agile Management User Manual

[Official website video]

[Agile Management Video]

The development of

The corresponding functions of development include development pipeline and application management. The main function is to create management applications, as well as the use of branch model and version control when developing applications. Software engineers’ daily development process operations are carried out here.

Information:

Create a project

Create an Nginx application

Create a back-end application

Create a front-end application

Application Management User Manual

Develop pipeline user manual

[Official website video]

[Development video]

test

Test management provides users with agile continuous testing tools, including test case management, test cycle, test analysis and so on, which can effectively improve the efficiency and quality of software testing, and improve the flexibility and visualization level of testing.

Information:

Using Test Management

Test Management User manual

[Official website video]

Continuous release and deployment

Using deployment assembly line, users can easily manage all sorts of use customer Choerodon application development and deployment of services, including application of start-stop, condition monitoring, and use the corresponding version control, container management, etc., at the same time also includes the application involves a variety of resource management, such as Internet, domain names, database services, cache service, etc.

Information:

Deployment pipeline user manual

[Official website video]

[Continuous delivery of video]

About the Choerodon toothfish

Choerodon is an open source enterprise services platform that builds on Kubernetes’ container orchestration and management capabilities and integrates DevOps toolchains, microservices and mobile application frameworks to help enterprises achieve agile application delivery and automated operations management. It also provides IoT, payment, data, intelligent insights, enterprise application marketplace and other business components to help enterprises focus on their business and accelerate digital transformation.

You can learn about the latest developments of toothfish, product features, and participate in community contributions through the following community channels:

• Liverpoolfc.tv: choerodon. IO
• BBS: forum. Choerodon. IO
• Github：github.com/choerodon/
• Choerodon toothfish
• The Choerodon toothfish

Welcome to join the Choerodon Toothfish community to create an open ecological platform for enterprise digital services.