This is my 10th day of the August Challenge.

Ansible Automation: The Implementation of Operating System Initialization implements the delivery of the operating system. Then we need to customize various basic software according to the requirements of research, development and operation and maintenance for the system online in the future. Here, my implementation method is to use Ansible Playbook to carry out the custom installation of basic software. In the future, Jenkins can also be combined to carry out on-demand continuous integration.

demand

Basic software requirements typically have the following scenarios:

  • Java project

JDK, Tomcat, application log directory, environment variables, and other customization requirements;

  • Python project

Python3, Anaconda, Supervisor, PIP source, etc.

  • nginx 

    Nginx custom-made;

  • ELK Log collection

    Filebeat collects relevant app logs;

  • The database

    MySQL compilation and installation, active synchronization, etc.

  • Web

LAMP, Vue and other environment configuration;

All the above are possible in the process of operation and maintenance. Therefore, we need to implement automatic installation in strict accordance with the directory specifications mentioned in Operation and Maintenance Considerations: The Importance of Directory Management Norms to avoid the differences in the manual installation process.

planning

With the PlayBook tag, we can categorize the requirements, such as:

  • nginx

Implement nGINX-related environment deployment

  • java

Implement java-related environment deployment

  • python

Implement Python-related environment deployment

  • filebeat

Collect logs

  • mysql

Deploy the mysql environment

  • lamp

    Implement LAMP related environment deployment

In this way, we can break down the complex environment deployment into small modules, simplify the complexity, one by one.

The specific implementation

Here we implement a custom installation of FileBeat, Java, Nginx, and Python.

1. Directory structure

Ansible PlayBook stores YML files in the following directory structure.

Roles /software_install/ ├─ files │ ├─ filebeat │ ├─ filebeat-6.43.- x86_64. RPM │ │ └ ─ ─ filebeat. Yml │ ├ ─ ─ Java │ │ ├ ─ ─ app. Tar. Gz │ │ ├ ─ ─ the app.zip│ │ ├ ─ ─ the data.zip│ │ └ ─ ─ jdk18.. 0 _40.zip│ ├─ Heavy Metal Guitar School │ ├─ Heavy Metal Guitar School - Heavy Metal Band.zip│ │ ├ ─ ─ nginx - module - VTS - master.zip│ │ ├ ─ ─ nginx_upstream_check_module - master.zip│ │ └ ─ ─ tengine -2.32..tar.gz │ ├ ── python │ ├ ── python -3.7. 0.tgz ├ ─ ─ handlers │ └ ─ ─ the main, yml ├ ─ ─ the tasks │ ├ ─ ─ filebeat. Yml │ ├ ─ ─ Java. Yml │ ├ ─ ─ the main, yml │ ├ ─ ─ nginx. Yml │ ├ ─ ─ ├ ─ ch.pdf │ ├ ─ ch.pdfvars└ ─ ─ the main ymlCopy the code

2. Define variables from VARS

During the entire installation process, we define the installation directory and dependencies of each component in the form of variables, which can be flexibly changed according to the actual situation.

# Overall custom variables
# var/main.yml
soft_dir: /usr/local/src
#filebeat
filebeat: filebeat-6.43.-x86_64.rpm

#nginx
nginx: tengine-2.32.
dependency:
  - libxslt
  - libxslt-devel
  - libxml2
  - libxml2-devel
  - gd-devel
  - geoip-devel

#java
jdk_path: /app/jdk18.. 0 _40
jdk_link_path: /app/jdk

#python
python: Python-3.7. 0
depend:
  - libffi-devel
  - zlib-devel
  - bzip2-devel
  - openssl-devel
  - ncurses-devel
  - sqlite-devel
  - readline-devel
  - tk-devel
  - gdbm-devel
  - db4-devel
  - libpcap-devel
  - xz-devel
  - zlib
  - zlib-devel
Copy the code

3. Task

1.filebeat.yml

The log directory and format that FileBeat collects should be defined in advance, so only the installation collection is done at this stage.

If fileBeat exists, do not install it
- name: test filebeat exist
  shell: which filebeat
  register: result
  ignore_errors: yes
  tags: filebeat

- name: copy filebeat to dest
  copy:
    src: "filebeat/{{ filebeat }}"
    dest: "{{ soft_dir }}"when: result.rc ! =0
  tags: filebeat

- name: install filebeat
  yum:
    name: "{{ soft_dir}}/{{ filebeat }}"state: present when: result.rc ! =0
  tags: filebeat

- name: config filebeat
  copy: 
    src: "filebeat/filebeat.yml"
    dest: "/etc/filebeat/filebeat.yml"
    backup: Truenotify: - restart filebeat when: result.rc ! =0tags: filebeat - name: filebeat enable service: name: filebeat enabled: true when: result.rc ! =0
  tags: filebeat
Copy the code

2.java.yml

Before installation, we can determine whether Java has been installed in advance to avoid repeated installation and improve efficiency.

# Copy JDK and standard directory to /app
- name: copy jdk to /app
  unarchive:
    src: "{{ item.src }}" 
    dest: "{{ item.dest }}"
    owner: hcuser
    group: wheel
  with_items:
    - { src: "java/app.zip", dest: "/" }
    - { src: "java/data.zip", dest: "/" }
    - { src: "Java/jdk1.8.0 _40.zip", dest: "/app" }
  tags: java

Check whether Java is installed
- name: test jdk exist
  shell: source /etc/profile && which java
  register: result
  ignore_errors: yes
  tags: java

# Java does not exist, soft link is set
- name: Create a symbolic link
  file:
    src: "{{ jdk_path }}"
    dest: "{{ jdk_link_path }}"state: link when: result.rc ! =0
  tags: java

If Java does not exist, set the Java environment variable
- name: set java profile
  lineinfile: 
    path: /etc/profile 
    line: "{{ item }}"
  with_items:
    - "#java"
    - "export JAVA_HOME={{ jdk_link_path }}"
    - "export JRE_HOME=$JAVA_HOME/jre"
    - "export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:./"
    - "export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH"when: result.rc ! =0
  tags: java
Copy the code

3.nginx.yml

Nginx plugins have been prepared in advance and are compiled and installed in a way that is more convenient for subsequent extensions than yum.

- name: copy nginx to dest
  unarchive:
    src: "{{ item }}"
    dest: "{{ soft_dir }}"
  with_items:
    - "nginx/echo-nginx-module-master.zip"
    - "nginx/nginx-module-vts-master.zip"
    - "nginx/nginx_upstream_check_module-master.zip"
    - "nginx/{{ nginx }}.tar.gz"
  tags: nginx

- name: install dependency
  yum:
    name: "{{ dependency }}"
    state: present
  tags: nginx

- name: install nginx
  shell: |
    cd  "{{ soft_dir }}/{{ nginx }}"
    ./configure --prefix=/app/nginx --add-module={{ soft_dir }}/nginx_upstream_check_module-master --add-module={{ soft_dir }}/nginx-module-vts-master/ --add-dynamic-module={{ soft_dir }}/echo-nginx-module-master   --with-http_ssl_module  --with-http_v2_module   --with-http_realip_module   --with-http_addition_module   --with-http_xslt_module   --with-http_xslt_module=dynamic  --with-http_image_filter_module  --with-http_image_filter_module=dynamic --with-http_geoip_module    --with-http_geoip_module=dynamic  --with-http_sub_module  --with-http_dav_module  --with-http_flv_module   --with-http_mp4_module --with-http_gunzip_module  --with-http_gzip_static_module   --with-http_auth_request_module   --with-http_random_index_module   --with-http_secure_link_module   --with-http_degradation_module      --with-http_slice_module    --with-http_stub_status_module   --with-stream    --with-stream=dynamic   --with-stream_ssl_module    --with-stream_realip_module --with-stream_geoip_module  --with-stream_geoip_module=dynamic --with-stream_ssl_preread_module  --with-stream_sni
    make
    make install
  tags: nginx
Copy the code

4.python.yml

Here we compile and install directly, but of course we can also install Miniconda to facilitate subsequent python multi-environment requirements.


- name: copy python to dest
  unarchive:
    src: "{{ item }}"
    dest: "{{ soft_dir }}"
  with_items:
    - "Python/python - 3.7.0..tgz"
  tags: python

- name: install depend
  yum:
    name: "{{ depend }}"
    state: present
  tags: python

- name: install python
  shell: |
    cd  "{{ soft_dir }}/{{ python }}"./configure --prefix=/usr/local/python3; make; make install tags: pythonCopy the code

5.main.yml

This file is the component to be installed.

- include: filebeat.yml
- include: nginx.yml
- include: java.yml
- include: python.yml
Copy the code

4.handlers

Handlers are used to restart the FileBeat service after the configuration file is changed so that the configuration file can be reloaded.

# handlers/main.yml
- name: restart filebeat
  service:
    name: filebeat
    state: restarted
Copy the code

5.files

The files directory is the source directory required for the installation of each component and is used for synchronization to the remote installation directory. For details, see the directory structure.

perform

With Tag we have the flexibility to install components on demand, or to batch install multiple services.

Install all software
ansible-playbook -b -e host_ip=10.102.10. -v sofware_install.yml
# Install FileBeat with tag
ansible-playbook -b -e host_ip=10.102.10. -v sofware_install.yml -t filebeat
Install nginx using tag
ansible-playbook -b -e host_ip=10.102.10. -v sofware_install.yml -t nginx
Install Python using tag
ansible-playbook -b -e host_ip=10.102.10. -v sofware_install.yml -t python
Install Java using tag
ansible-playbook -b -e host_ip=10.102.10. -v sofware_install.yml -t java
Install all software on multiple servers at the same time
ansible-playbook -b -e host_ip=10.102.10..10.102.11. -v sofware_install.yml
Copy the code

conclusion

The basic software installation phase depends on the “directory management specifications”, as the saying goes “no rules, no circle”, we hope that each team member has the basic environment like control before logging in to the server, so as to improve our work efficiency invisibly.

Do you think this is the end of our basic software installation? Next, we will continue to add Jenkins to achieve graphical control installation. Please stay tuned!