Apache2.4 + django + windows10 automatic deployment

Set up a Webhooks URL route in your Django project, run the automatic deployment script asynchronously on that route, access the route and execute the script successfully without problems when not deployed

The script involves pulling the update local code and restarting apache2.4

After apache2.4 deployed djano, there were no problems with project access, but the following problems were encountered when executing automated deployment scripts

Executable could not find the path to the Python interpreter in the virtual environment. Sys. executable returns the path to httpd.exe

Cannot be imported in automated scriptsdjangoPackage files in the project can only be imported currentlypythonPackage files in the environment!

Environment variables executed in automated scripts are different than expected%USERPROFILE%The path becomesC:\WINDOWS\system32\config\systemprofile

The current could not be found after apache2.4 deployment`python`Path to the interpreter

Bug emersion
```
 Outputs sys.executable in any Django view
 def test(request""" Test view ""logger.info("sys.executable: {}".format(sys.executable))

Copy the code
```
Then restart Apache to access the view and you’ll be surprised as it outputs: c:\apache24\bin\httpd.exe

why????

Confused, I simply printed sys.path to try to find the Python path

But it outputs the package file path and the project root path in the virtual environment

Like this: e:\development\django_project\env\Lib\site-packages
The solution

Use \Scripts\python.exe to replace \Lib\site-packages with \Lib\site-packages

Git pull command error in automatic script. Procedure

The error message is as follows

Host key verification failed. fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.

Say I do not have permission, how can I just push a Baidu found after the problem seems to boil down to.ssh directory problem

Git will set up an SSH connection to retrieve the ssh_key from the current user directory

So I just printed the value of %USERPROFILE% this time

Sure enough, after Apache deployment %USERPROFILE% became C:\WINDOWS\ System32 \config\ SystemProfile

Ah! This? I didn’t know where to start, so I copied the current user’s.shh directory to the above path

Restart The Apache script

Custom package files for introducing Django projects into automated scripts

I manually optimized the deployment script

Extract some keys into conf\env.py for unified management

Manually starting the Django project runs flawlessly through Apache deployment and does not respond

I then added logger and found the package file was not found when importing env

Absolutely!!!!!! Well, the only way to load the key is to open a file

This time it worked perfectly!!

Subprocess. Py error

Ecstatically tested

Git commit -m "Fix: Test deployment script"

git push

Then I waited a while and nothing happened. I opened the update log and looked at it

Error:

[2021-11-14 02:27:01.467][root.<module>():119] [ERROR] Traceback (most recent call): File "E:\development\django_project\utils\auto_update.py", line 117, in <module> update() File "E:\development\django_project\utils\auto_update.py", line 88, in update flag, git_res = git_pull() File "E:\development\django_project\utils\auto_update.py", line 55, in git_pull messages = exec_cmd(message_list) File "E:\development\django_project\utils\util.py", line 54, in exec_cmd out, err = proc.communicate(timeout=timeout) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\Lib\subprocess.py", line 964, in communicate stdout, stderr = self._communicate(input, endtime, timeout) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\Lib\subprocess.py", line 1317, in _communicate stdout = stdout[0] IndexError: list index out of rangeCopy the code

The relevant code is as follows: I added the comment part later


 # django_project\utils\auto_update.py
 from pip._internal import main as pip
 def exec_cmd(cmd: str, timeout=10) :
     proc = subprocess.Popen(cmd,
                             shell=True,
                             stdout=subprocess.PIPE,
                             stderr=subprocess.STDOUT,
                             encoding='gbk')
     res = []
     try:
         out, err = proc.communicate(timeout=timeout)
     except subprocess.TimeoutExpired as ex:
         res.append("commend_timeout:{}".format(ex))
         return res
     for line in out.splitlines():
         res.append(line)
 
     return res
 
 def git_pull() :
     """ git pull :return: """
     pull = 'git -C {} pull --force origin master:master'.format(project_path)
     change_file_list = 'git -C {} diff --name-status HEAD@{{1}}.. HEAD'.format(project_path)
     message_list = 'git -C {} log HEAD@{{1}}.. HEAD --pretty=format:"%s from %cn at %ci"'.format(project_path)
     res_lines = exec_cmd(pull, 20)
     logging.info(res_lines)
     res_lines.insert(0.'1. git pull')
     if 'Already up to date.'! = res_lines[-1] :return False, res_lines
     change_file = exec_cmd(change_file_list, 20)
     logging.info(change_file)
     change_file.insert(0.'2. change file list')
     res_lines.extend(change_file)
     # time.sleep(1) 
     # try:
     messages = exec_cmd(message_list, 20)
     # logging.info(messages)
     messages.insert(0.'3. push message')
     res_lines.extend(messages)
     # except Exception:
     # logging.error('message: failed to fetch: {}'. Format (traceback.format_exc()))
     for item in change_file:
         if 'requirments.txt' in item:
             pip(['install'.'-r', os.path.join(project_path, 'requirments.txt'), '-i'.'https://pypi.douban.com/simple'])
             break
     return True, res_lines
 
Copy the code

I then added the comment section to logger and found this output [2021-11-14 02:54:18.058][root.git_pull():60] [INFO] [‘style: � � � � � ° � � Զ � � � � � the from bangenlanbai 02:17:20 at 2021-11-14 + 0800 ‘]

Chinese??????? commit ? Proc =subprocess. Popen(CMD, shell=True, stdout=subprocess.PIPE, stderr=subprocess. stdout, encoding=’ GBK ‘)

Net stop apache2.4 / Net start Apache2.4 / net stop apache2.4

So I changed this function

 def exec_cmd(cmd: str, timeout=10) :
     proc = subprocess.Popen(cmd,
                             shell=True,
                             stdout=subprocess.PIPE,
                             stderr=subprocess.STDOUT,
                             encoding=None) Return byte output if encoding is not passed
     res = []
     try:
         out, err = proc.communicate(timeout=timeout)
     except subprocess.TimeoutExpired as ex:
         res.append("commend_timeout:{}".format(ex))
         return res
     for line in out.splitlines():
         # line is now of type bytes
         try:
             line_str = str(line, encoding='GBK')  # Try coding here
         except Exception:
             line_str = str(line, encoding='utf-8')
         res.append(res)
     return res
Copy the code

After the script runs perfectly!

There is one small problem

Python’s default logging to files causes GBK encoding problems

 The default log configuration file format is GBK
 logging.basicConfig(level=logging.INFO,
                     filename=log_path,
                     filemode='a'.format=
                     '[%(asctime)s][%(name)s.%(funcName)s():%(lineno)d] [%(levelname)s] %(message)s'.)# Use logging.fileHandler (filename=log_path, encoding=' UTF-8 ') to specify the file encoding
 logger = logging.getLogger('auto_update')
 file_handel = logging.FileHandler(filename=log_path, encoding='utf-8')
 logger.setLevel('INFO')
 file_handel.setFormatter(
     logging.Formatter('[%(asctime)s][%(name)s.%(funcName)s():%(lineno)d] [%(levelname)s] %(message)s'))
 logger.addHandler(file_handel)

Copy the code

The improved deployment script is as follows

 # -*- coding:utf-8 -*-
 # @Time : 2021/11/12 19:32
 # @Author : BGLB
 # @Software : PyCharm
 # auto_update.py 
 
 import datetime
 import logging
 import os
 import subprocess
 import sys
 import time
 import traceback
 
 from dingtalkchatbot.chatbot import DingtalkChatbot
 from pip._internal import main as pip
 
 self_path = os.path.abspath(__file__)
 
 project_path = os.path.dirname(os.path.dirname(self_path))
 log_path = os.path.join(project_path, 'logs'.'auto_update.log')
 
 logger = logging.getLogger('auto_update')
 file_handel = logging.FileHandler(filename=log_path, encoding='utf-8')
 logger.setLevel('INFO')
 file_handel.setFormatter(
     logging.Formatter('[%(asctime)s][%(name)s.%(funcName)s():%(lineno)d] [%(levelname)s] %(message)s'))
 logger.addHandler(file_handel)
 
 
 def exec_cmd(cmd: str, timeout=10) :
     proc = subprocess.Popen(cmd,
                             shell=True,
                             stdout=subprocess.PIPE,
                             stderr=subprocess.STDOUT,
                             encoding=None)
     res = []
     try:
         out, err = proc.communicate(timeout=timeout)
     except subprocess.TimeoutExpired as ex:
         res.append("commend_timeout:{}".format(ex))
         return res
     for line in out.splitlines():
         try:
             line_str = line.decode()
         except Exception:
             line_str = line.decode('GBK')
 
         res.append(line_str)
 
     return res
 
 
 def start_auto_update() :
     """ Start yourself :return: """
     python_path = sys.executable if sys.executable.endswith('python.exe') else sys.path[0].replace('Lib\site-packages'.'Scripts\python.exe')
     os.system('start "auto_update" /d {} {} {}'.format(project_path, python_path, self_path))
 
 
 def git_pull() :
     """ git pull :return: """
     pull = 'git -C {} pull --force origin master:master'.format(project_path)
     change_file_list = 'git -C {} diff --name-status HEAD@{{1}}.. HEAD'.format(project_path)
     message_list = 'git -C {} log HEAD@{{1}}.. HEAD --pretty=format:"%s from %cn at %ci"'.format(project_path)
     res_lines = exec_cmd(pull, 20)
     logger.info(res_lines)
     res_lines.insert(0.'1. git pull')
     if 'Already up to date.'! = res_lines[-1] :return False, res_lines
     change_file = exec_cmd(change_file_list, 20)
     logger.info(change_file)
     change_file.insert(0.'2. change file list')
     res_lines.extend(change_file)
     time.sleep(1)
     messages = exec_cmd(message_list, 20)
     logging.info(messages)
     messages.insert(0.'3. push message')
     res_lines.extend(messages)
     for item in change_file:
         if 'requirments.txt' in item:
             try:
                 logger.info('download requirments. TXT')
                 pip(['install'.'-r', os.path.join(project_path, 'requirments.txt'), '-i'.'https://pypi.douban.com/simple'])
                 break
             except Exception:
                 log_str = F "Download failed: \n{traceback.format_exc()}"
                 logger.error(log_str)
                 res_lines.append(log_str)
                 return False, res_lines
     return True, res_lines
 
 
 def restart_apache() :
     """ Restart Apache :return: """
 
     cmd_line = ['net stop apache2.4'.'net start apache2.4']
     res = []
     for cmd in cmd_line:
         res.extend(exec_cmd(cmd, 60*2))
         time.sleep(1)
     if 'apache2.4 service started successfully. ' in res:
         return True, res
     return False, res
 
 
 def update() :
     """ update :return: ""
     # from conf.env import DINGDING_ROBOT_URL, DINGDING_SECRET, DINGDING_AT_MOBILES
     DINGDING_ROBOT_URL = ' '
     DINGDING_SECRET = ' '
     DINGDING_AT_MOBILES = []
     It is still a bit inelegant to read env configuration
     env_path = os.path.join(project_path, 'conf'.'env.py')
     with open(env_path, encoding='utf=8', mode='r') as f:
         text = f.readlines()
     for item in text:
         if item and item.startswith('DINGDING'):
             value = item.strip(' ').strip('\n').split('='.1)[-1]
 
             if 'DINGDING_ROBOT_URL' in item:
                 DINGDING_ROBOT_URL = eval(value)
 
             if item and 'DINGDING_SECRET' in item:
                 DINGDING_SECRET = eval(value)
 
             if item and 'DINGDING_AT_MOBILES' in item:
                 DINGDING_AT_MOBILES = eval(value)
 
     dingding = DingtalkChatbot(DINGDING_ROBOT_URL, secret=DINGDING_SECRET)
     flag, git_res = git_pull()
     markdown = '# Automatic deployment log \n\nstart_time: {}\n\n'.format(datetime.datetime.now().strftime('%Y-%m-%d ''%H:%M:%S'))
     markdown += '\n\n**pull update**\n'
     for index, item in enumerate(git_res):
         prefix = ' ' if item.endswith('</span>') else > '-'
         markdown += '\n{}{}\n'.format(prefix, item.lstrip())
 
     if not flag:
         markdown += '\ n \ n \ n * * < span style = "color: red" > the git pull failure < / span > to go take a look at * * \ n'
         dingding.send_markdown(title='Automated deployment failed', text=markdown, at_mobiles=DINGDING_AT_MOBILES)
         return
 
     flag, restart_apache_res = restart_apache()
     markdown += '\n\n\n**restart apache**\n>\n'
     for item in restart_apache_res:
         if item.strip(' '):
             markdown += '> - {}\n'.format(item.lstrip(' '))
 
     if not flag:
         markdown += '\n\n\n** Restart Apache failed 
         dingding.send_markdown(title='Automated deployment failed', text=markdown, at_mobiles=DINGDING_AT_MOBILES)
         return
 
     markdown += '\n\n\nend_time: {}\n\n\n'.format(datetime.datetime.now().strftime('%Y-%m-%d ''%H:%M:%S'))
     dingding.send_markdown(title='Automated deployment successful', text=markdown, at_mobiles=DINGDING_AT_MOBILES)
 
 # Markdown syntax does not seem to support HTML syntax
 if __name__ == '__main__':
     try:
         update()
     except Exception:
         logger.error(traceback.format_exc())
 
Copy the code

The project structure

─ django_project │ the manage. Py │ requirments. TXT ├ ─ the conf │ │ env. Py ├ ─ utils │ │ auto_update. Py ├ ─ logs │ │... ├─ Django_project │ │ ├─ Test_App │ │ view.py │ │ models.py │ urls.pyCopy the code

Apache2.4 + django + windows10 automatic deployment

Apache2.4 + django + windows10 automatic deployment

The current could not be found after apache2.4 deploymentpythonPath to the interpreter

Git pull command error in automatic script. Procedure

Custom package files for introducing Django projects into automated scripts

Subprocess. Py error

The improved deployment script is as follows

Related Posts

Hongmeng light kernel source code analysis: virtual real mapping

Rainy day: the goddess said that there are stars in the evening to date with me? Python will help you draw a sky full of stars!

Unified access and convergence management of security surveillance video through LiveGBS/LiveNVR (GB28181, Onvif/RTSP, etc.)

The current could not be found after apache2.4 deployment`python`Path to the interpreter