Apache2.4 + django + windows10 automatic deployment
Set up a Webhooks URL route in your Django project, run the automatic deployment script asynchronously on that route, access the route and execute the script successfully without problems when not deployed
The script involves pulling the update local code and restarting apache2.4
After apache2.4 deployed djano, there were no problems with project access, but the following problems were encountered when executing automated deployment scripts
- Executable could not find the path to the Python interpreter in the virtual environment. Sys. executable returns the path to httpd.exe
- Cannot be imported in automated scripts
djangoPackage files in the project can only be imported currentlypythonPackage files in the environment!- Environment variables executed in automated scripts are different than expected
%USERPROFILE%The path becomesC:\WINDOWS\system32\config\systemprofile
The current could not be found after apache2.4 deploymentpythonPath to the interpreter
-
Bug emersion
Outputs sys.executable in any Django view def test(request""" Test view ""logger.info("sys.executable: {}".format(sys.executable)) Copy the codeThen restart Apache to access the view and you’ll be surprised as it outputs: c:\apache24\bin\httpd.exe
why????
Confused, I simply printed sys.path to try to find the Python path
But it outputs the package file path and the project root path in the virtual environment
Like this: e:\development\django_project\env\Lib\site-packages
-
The solution
Use \Scripts\python.exe to replace \Lib\site-packages with \Lib\site-packages
Git pull command error in automatic script. Procedure
The error message is as follows
Host key verification failed. fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.
Say I do not have permission, how can I just push a Baidu found after the problem seems to boil down to.ssh directory problem
Git will set up an SSH connection to retrieve the ssh_key from the current user directory
So I just printed the value of %USERPROFILE% this time
Sure enough, after Apache deployment %USERPROFILE% became C:\WINDOWS\ System32 \config\ SystemProfile
Ah! This? I didn’t know where to start, so I copied the current user’s.shh directory to the above path
Restart The Apache script
Custom package files for introducing Django projects into automated scripts
I manually optimized the deployment script
Extract some keys into conf\env.py for unified management
Manually starting the Django project runs flawlessly through Apache deployment and does not respond
I then added logger and found the package file was not found when importing env
Absolutely!!!!!! Well, the only way to load the key is to open a file
This time it worked perfectly!!
Subprocess. Py error
Ecstatically tested
Git commit -m "Fix: Test deployment script"
git push
Then I waited a while and nothing happened. I opened the update log and looked at it
Error:
[2021-11-14 02:27:01.467][root.<module>():119] [ERROR] Traceback (most recent call): File "E:\development\django_project\utils\auto_update.py", line 117, in <module> update() File "E:\development\django_project\utils\auto_update.py", line 88, in update flag, git_res = git_pull() File "E:\development\django_project\utils\auto_update.py", line 55, in git_pull messages = exec_cmd(message_list) File "E:\development\django_project\utils\util.py", line 54, in exec_cmd out, err = proc.communicate(timeout=timeout) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\Lib\subprocess.py", line 964, in communicate stdout, stderr = self._communicate(input, endtime, timeout) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\Lib\subprocess.py", line 1317, in _communicate stdout = stdout[0] IndexError: list index out of rangeCopy the codeThe relevant code is as follows: I added the comment part later
# django_project\utils\auto_update.py
from pip._internal import main as pip
def exec_cmd(cmd: str, timeout=10) :
proc = subprocess.Popen(cmd,
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
encoding='gbk')
res = []
try:
out, err = proc.communicate(timeout=timeout)
except subprocess.TimeoutExpired as ex:
res.append("commend_timeout:{}".format(ex))
return res
for line in out.splitlines():
res.append(line)
return res
def git_pull() :
""" git pull :return: """
pull = 'git -C {} pull --force origin master:master'.format(project_path)
change_file_list = 'git -C {} diff --name-status HEAD@{{1}}.. HEAD'.format(project_path)
message_list = 'git -C {} log HEAD@{{1}}.. HEAD --pretty=format:"%s from %cn at %ci"'.format(project_path)
res_lines = exec_cmd(pull, 20)
logging.info(res_lines)
res_lines.insert(0.'1. git pull')
if 'Already up to date.'! = res_lines[-1] :return False, res_lines
change_file = exec_cmd(change_file_list, 20)
logging.info(change_file)
change_file.insert(0.'2. change file list')
res_lines.extend(change_file)
# time.sleep(1)
# try:
messages = exec_cmd(message_list, 20)
# logging.info(messages)
messages.insert(0.'3. push message')
res_lines.extend(messages)
# except Exception:
# logging.error('message: failed to fetch: {}'. Format (traceback.format_exc()))
for item in change_file:
if 'requirments.txt' in item:
pip(['install'.'-r', os.path.join(project_path, 'requirments.txt'), '-i'.'https://pypi.douban.com/simple'])
break
return True, res_lines
Copy the codeI then added the comment section to logger and found this output [2021-11-14 02:54:18.058][root.git_pull():60] [INFO] [‘style: � � � � � ° � � Զ � � � � � the from bangenlanbai 02:17:20 at 2021-11-14 + 0800 ‘]
Chinese??????? commit ? Proc =subprocess. Popen(CMD, shell=True, stdout=subprocess.PIPE, stderr=subprocess. stdout, encoding=’ GBK ‘)
Net stop apache2.4 / Net start Apache2.4 / net stop apache2.4
So I changed this function
def exec_cmd(cmd: str, timeout=10) :
proc = subprocess.Popen(cmd,
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
encoding=None) Return byte output if encoding is not passed
res = []
try:
out, err = proc.communicate(timeout=timeout)
except subprocess.TimeoutExpired as ex:
res.append("commend_timeout:{}".format(ex))
return res
for line in out.splitlines():
# line is now of type bytes
try:
line_str = str(line, encoding='GBK') # Try coding here
except Exception:
line_str = str(line, encoding='utf-8')
res.append(res)
return res
Copy the codeAfter the script runs perfectly!
There is one small problem
Python’s default logging to files causes GBK encoding problems
The default log configuration file format is GBK
logging.basicConfig(level=logging.INFO,
filename=log_path,
filemode='a'.format=
'[%(asctime)s][%(name)s.%(funcName)s():%(lineno)d] [%(levelname)s] %(message)s'.)# Use logging.fileHandler (filename=log_path, encoding=' UTF-8 ') to specify the file encoding
logger = logging.getLogger('auto_update')
file_handel = logging.FileHandler(filename=log_path, encoding='utf-8')
logger.setLevel('INFO')
file_handel.setFormatter(
logging.Formatter('[%(asctime)s][%(name)s.%(funcName)s():%(lineno)d] [%(levelname)s] %(message)s'))
logger.addHandler(file_handel)
Copy the codeThe improved deployment script is as follows
# -*- coding:utf-8 -*-
# @Time : 2021/11/12 19:32
# @Author : BGLB
# @Software : PyCharm
# auto_update.py
import datetime
import logging
import os
import subprocess
import sys
import time
import traceback
from dingtalkchatbot.chatbot import DingtalkChatbot
from pip._internal import main as pip
self_path = os.path.abspath(__file__)
project_path = os.path.dirname(os.path.dirname(self_path))
log_path = os.path.join(project_path, 'logs'.'auto_update.log')
logger = logging.getLogger('auto_update')
file_handel = logging.FileHandler(filename=log_path, encoding='utf-8')
logger.setLevel('INFO')
file_handel.setFormatter(
logging.Formatter('[%(asctime)s][%(name)s.%(funcName)s():%(lineno)d] [%(levelname)s] %(message)s'))
logger.addHandler(file_handel)
def exec_cmd(cmd: str, timeout=10) :
proc = subprocess.Popen(cmd,
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
encoding=None)
res = []
try:
out, err = proc.communicate(timeout=timeout)
except subprocess.TimeoutExpired as ex:
res.append("commend_timeout:{}".format(ex))
return res
for line in out.splitlines():
try:
line_str = line.decode()
except Exception:
line_str = line.decode('GBK')
res.append(line_str)
return res
def start_auto_update() :
""" Start yourself :return: """
python_path = sys.executable if sys.executable.endswith('python.exe') else sys.path[0].replace('Lib\site-packages'.'Scripts\python.exe')
os.system('start "auto_update" /d {} {} {}'.format(project_path, python_path, self_path))
def git_pull() :
""" git pull :return: """
pull = 'git -C {} pull --force origin master:master'.format(project_path)
change_file_list = 'git -C {} diff --name-status HEAD@{{1}}.. HEAD'.format(project_path)
message_list = 'git -C {} log HEAD@{{1}}.. HEAD --pretty=format:"%s from %cn at %ci"'.format(project_path)
res_lines = exec_cmd(pull, 20)
logger.info(res_lines)
res_lines.insert(0.'1. git pull')
if 'Already up to date.'! = res_lines[-1] :return False, res_lines
change_file = exec_cmd(change_file_list, 20)
logger.info(change_file)
change_file.insert(0.'2. change file list')
res_lines.extend(change_file)
time.sleep(1)
messages = exec_cmd(message_list, 20)
logging.info(messages)
messages.insert(0.'3. push message')
res_lines.extend(messages)
for item in change_file:
if 'requirments.txt' in item:
try:
logger.info('download requirments. TXT')
pip(['install'.'-r', os.path.join(project_path, 'requirments.txt'), '-i'.'https://pypi.douban.com/simple'])
break
except Exception:
log_str = F "Download failed: \n{traceback.format_exc()}"
logger.error(log_str)
res_lines.append(log_str)
return False, res_lines
return True, res_lines
def restart_apache() :
""" Restart Apache :return: """
cmd_line = ['net stop apache2.4'.'net start apache2.4']
res = []
for cmd in cmd_line:
res.extend(exec_cmd(cmd, 60*2))
time.sleep(1)
if 'apache2.4 service started successfully. ' in res:
return True, res
return False, res
def update() :
""" update :return: ""
# from conf.env import DINGDING_ROBOT_URL, DINGDING_SECRET, DINGDING_AT_MOBILES
DINGDING_ROBOT_URL = ' '
DINGDING_SECRET = ' '
DINGDING_AT_MOBILES = []
It is still a bit inelegant to read env configuration
env_path = os.path.join(project_path, 'conf'.'env.py')
with open(env_path, encoding='utf=8', mode='r') as f:
text = f.readlines()
for item in text:
if item and item.startswith('DINGDING'):
value = item.strip(' ').strip('\n').split('='.1)[-1]
if 'DINGDING_ROBOT_URL' in item:
DINGDING_ROBOT_URL = eval(value)
if item and 'DINGDING_SECRET' in item:
DINGDING_SECRET = eval(value)
if item and 'DINGDING_AT_MOBILES' in item:
DINGDING_AT_MOBILES = eval(value)
dingding = DingtalkChatbot(DINGDING_ROBOT_URL, secret=DINGDING_SECRET)
flag, git_res = git_pull()
markdown = '# Automatic deployment log \n\nstart_time: {}\n\n'.format(datetime.datetime.now().strftime('%Y-%m-%d ''%H:%M:%S'))
markdown += '\n\n**pull update**\n'
for index, item in enumerate(git_res):
prefix = ' ' if item.endswith('</span>') else > '-'
markdown += '\n{}{}\n'.format(prefix, item.lstrip())
if not flag:
markdown += '\ n \ n \ n * * < span style = "color: red" > the git pull failure < / span > to go take a look at * * \ n'
dingding.send_markdown(title='Automated deployment failed', text=markdown, at_mobiles=DINGDING_AT_MOBILES)
return
flag, restart_apache_res = restart_apache()
markdown += '\n\n\n**restart apache**\n>\n'
for item in restart_apache_res:
if item.strip(' '):
markdown += '> - {}\n'.format(item.lstrip(' '))
if not flag:
markdown += '\n\n\n** Restart Apache failed
dingding.send_markdown(title='Automated deployment failed', text=markdown, at_mobiles=DINGDING_AT_MOBILES)
return
markdown += '\n\n\nend_time: {}\n\n\n'.format(datetime.datetime.now().strftime('%Y-%m-%d ''%H:%M:%S'))
dingding.send_markdown(title='Automated deployment successful', text=markdown, at_mobiles=DINGDING_AT_MOBILES)
# Markdown syntax does not seem to support HTML syntax
if __name__ == '__main__':
try:
update()
except Exception:
logger.error(traceback.format_exc())
Copy the codeThe project structure
─ django_project │ the manage. Py │ requirments. TXT ├ ─ the conf │ │ env. Py ├ ─ utils │ │ auto_update. Py ├ ─ logs │ │... ├─ Django_project │ │ ├─ Test_App │ │ view.py │ │ models.py │ urls.pyCopy the code