Unified management of hooks using pre-commit in Git projects

The Unix philosophy

Offer “sharp” gadgets, each designed to do one thing well.

— How to Become a Programmer — From engineer to Expert

Writing in the front

If you use Git, you know the power of plain text and love scripting languages like the shell.

Most of the time, I prefer tools that can be configured in a scripting language over tools that are installed directly into the editor. One is that scripts can be shared with more people in a project to keep the specifications in place. Second, there is no need to remember more shortcut keys or click a mouse when the script automatically triggers the operation; The third is that scripting languages can do much more flexibly than software developers. This is probably one of the Git tools I’ve always preferred to use with Git instructions, rather than the compiler.

This article continues with git hooks, introducing a tool that helps you better manage and leverage Them. The tool you want to find has the following functions:

• Simply provide the configuration file to automatically get the script from the central hooks repository
  • If you have multiple projects, you no longer need to have a copy of hooks for each project
• Local script repositories can be defined, allowing developers to customize scripts without modifying configuration files
  • Developers will have scripts to perform custom actions
  • No need to modify a configuration file means that you can either point directly to a directory and execute all of the hooks in it, or specify a local configuration file that does not need to be uploaded to Git
• Multiple scripts can be defined per stage
  • Multiple scripts allow functionality to be partitioned without having to consolidate into a bloated file
• Scripts support multiple languages

The pre – commit profile

Do not be fooled by the name “pre-commit”. This tool works not only on the “pre-commit” stage, but also on any stage set to git-hooks. See “Stages configuration” for details. (The name probably comes from the fact that they only did the pre-commit phase at first, and then expanded to other phases.)

Install the pre – commit

Install in the systempre-commit

brew install pre-commit
# or
pip install pre-commit

# check version
pre-commit --version
# pre-commit 2.12.1 <- This is the version I'm currently using
Copy the code

Install in the projectpre-commit

cd <git-repo>
pre-commit install
# uninstall
pre-commit uninstall
Copy the code

This will generate a pre-commit file in the.git/hooks directory of the project (overwriting the original pre-commit file), which will perform tasks based on the.pre-commit-config.yaml directory of the project root. If you can see the implementation of the code in vim.git /hooks/pre-commit, the basic logic is to use the pre-commit file to extend more pre-commits, which is similar to the logic in my last article.

Install/uninstall hooks for other phases.

pre-commit install
pre-commit uninstall
-t {pre-commit,pre-merge-commit,pre-push,prepare-commit-msg,commit-msg,post-checkout,post-commit,post-merge}
--hook-type {pre-commit,pre-merge-commit,pre-push,prepare-commit-msg,commit-msg,post-checkout,post-commit,post-merge}

--hook-type prepare-commit- MSG
Copy the code

Commonly used instructions

# Manually implement hooks on all files. Executing this directive directly does not require the pre-commit phase to trigger hooks
pre-commit run --all-files
# Perform specific hooks
pre-commit run <hook_id>
Update all hooks to the latest version /tag
pre-commit autoupdate
Specifies the repO to be updated
pre-commit autoupdate --repo https://github.com/DoneSpeak/gromithooks
Copy the code

For more instructions and parameters, visit the official website of pre-commit.

Add third-party hooks

cd <git-repo>
pre-commit install
touch .pre-commit-config.yaml
Copy the code

The following is a basic configuration example.

.pre-commit-config.yaml

# This config file is the pre-commit configuration file for the project, which specifies which Git hooks can be executed for the project

# This is one of the global configurations for pre-commit
fail_fast: false

repos:
# the warehouse where hook is located
- repo: https://github.com/pre-commit/pre-commit-hooks
  You can use tags or branches directly, but branches are subject to change
  # If you branch, it will not update automatically after the first installation
  # You can update the tag to the latest tag of the default branch by using the 'pre-commit autoupdate' directive
  rev: v4.0.1
  The hook ID in the repository
  hooks:
  The hook script is defined in repo's.pre-commit-links.yaml
  - id: check-added-large-files
  # Remove trailing Spaces
  - id: trailing-whitespace
    Makedown is not handled
    args: [--markdown-linebreak-ext=md]
  Check for merge conflict symbols
  - id: check-merge-conflict
- repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
  rev: v2.0.0
  hooks:
  - id: pretty-format-yaml
    # https://github.com/macisamuele/language-formatters-pre-commit-hooks/blob/v2.0.0/language_formatters_pre_commit_hooks/pre tty_format_yaml.py
    The parameters required by the # hook script can be seen in the hook script file
    args: [--autofix.--indent.'2']
Copy the code

After the run, pre-commit downloads the specified repository code and installs the runtime environment required for the configuration. After the configuration is complete, you can run the added hooks with pre-commit Run –all-files. The following table shows the. Pre-commit – links. yaml Optional configuration items.

Developing the Hooks repository

The hooks for using third parties in your project have been explained above, but some of the functionality is required for customization and not available from third parties. This is where we need to develop our own hooks repository.

As long as your git repo is an installable package (gem, npm, pypi, etc.) or exposes an executable, it can be used with pre-commit.

As long as your Git repository is installable or exposed as executable, it can be used by pre-commit. The project demonstrated here is a packable Python project. It was also the first time to write such a project, and it took a lot of effort. If you are not familiar with Python, you can either use Packaging Python Projects or use third-party hooks repositories.

Here is the basic directory structure of the project (see the source path for the complete project at the end of this article) :

├ ─ ─ the README. Md ├ ─ ─ pre_commit_hooks │ ├ ─ ─ just set py │ ├ ─ ─ cm_tapd_autoconnect. Py# The actual script executed│ ├ ─ ─ pcm_issue_ref_prefix. Py# The actual script executed│ └ ─ ─ pcm_tapd_ref_prefix. Py# The actual script executed├ ─ ─. Pre - commit - hooks. Yaml# configure pre-commit hooks entry├ ─ ─ pyproject. Toml ├ ─ ─ setup. The CFGConfigure the script executed by Hook Entry Point└ ─ ─ setup. PyCopy the code

A Git repository that contains the pre-commit plug-in must contain a. Pre-commit – links. yaml file that informs the pre-commit plug-in. Yaml configuration options are the same as.pre-commit-config.yaml.

.pre-commit-hooks.yaml

# This project is a pre-commit hooks repository project that provides hooks externally

- id: pcm-issue-ref-prefix
  name: Add issue reference prefix for commit msg
  description: Add issue reference prefix for commit msg to link commit and issue
  entry: pcm-issue-ref-prefix
  The language used to implement hook
  language: python
  stages: [prepare-commit-msg]
- id: pcm-tapd-ref-prefix
  name: Add tapd reference prefix for commit msg
  description: Add tapd reference prefix for commit msg
  entry: pcm-tapd-ref-prefix
  The language used to implement hook
  language: python
  stages: [prepare-commit-msg]
  Yaml: # force the output of intermediate logs. This is not configured, and is specified by the user in.pre-commit-config.yaml
  # verbose: true
- id: cm-tapd-autoconnect
  name: Add tapd reference for commit msg
  description: Add tapd reference for commit msg to connect tapd and commit
  entry: cm-tapd-autoconnect
  The language used to implement hook
  language: python
  stages: [commit-msg]
Copy the code

Where entry is the command to execute and corresponds to the list configured in [options.entry_points] in setup.cfg.

setup.cfg

. [options.entry_points]console_scripts = cm-tapd-autoconnect = pre_commit_hooks.cm_tapd_autoconnect:main pcm-tapd-ref-prefix = pre_commit_hooks.pcm_tapd_ref_prefix:main pcm-issue-ref-prefix = pre_commit_hooks.pcm_issue_ref_prefix:mainCopy the code

Here is the Python script for pcM-issue-ref-prefix, which is used to add a prepare-commit-msg hook that prefixes the issue to the commit message based on the branch name.

pre_commit_hooks/pcm_issue_ref_prefix.py

# Automatically add a COMMIT Message prefix to associate issue and COMMIT based on the branch name. # # # branch name | commit format -- - | -- - | # 1234, # issue - 1234 message# issue - 1234 - fix - bug | # 1234, messageimport sys, OS, refrom subprocess import check_outputfrom typing import Optionalfrom typing import Sequencedef main(argv: Optional[Sequence[str]] = None) -> int: Branch = check_output(['git', 'symbolic-ref', '--short', 'HEAD']).strip().decode('utf-8') # Issue-123, issue-1234-fix result = re.match('^issue-(\d+)((-.*)+)?$', branch) if not result: # Branch name not matching warning = "WARN: Unable to add issue prefix since the format of the branch name dismatch." warning += "\nThe branch should look like issue-
      
        or issue-
       
        -
        
         , for example: issue-100012 or issue-10012-fix-bug)" print(warning) return issue_number = result.group(1) with open(commit_msg_filepath, 'r+') as f: content = f.read() if re.search('^#[0-9]+(.*)', content): # print('There is already issue prefix in commit message.') return issue_prefix = '#' + issue_number f.seek(0, 0) f.write("%s, %s" % (issue_prefix, content)) # print('Add issue prefix %s to commit message.' % issue_prefix)if __name__ == '__main__': exit(main())
        
       
      
Copy the code

Argv [1] is used to retrieve the path to the commit_msg file. Of course, you can also retrieve it with argparse. The parameters of some phases can be found in the install command description on the pre-commit website.

import argparsefrom typing import Optionalfrom typing import Sequencedef main(argv: Optional[Sequence[str]] = None) - >int:    parser = argparse.ArgumentParser()    parser.add_argument('filename', nargs=The '*'.help='Filenames to check.')    args = parser.parse_args(argv)    # .git/COMMIT_EDITMSG print("commit_msg file is " + args.filename[0])if __name__ == '__main__': exit(main())
Copy the code

You only need to perform the following configuration in the project to be configured: pre-commit-config.yaml

repos:
- repo: https://github.com/DoneSpeak/gromithooks
  rev: v1.0.0
  hooks:
  - id: pcm-issue-ref-prefix
    verbose: true
    # specify the stage of hook execution
    stages: [prepare-commit-msg]
Copy the code

Local hooks

Pre-commit also provides local hooks that allow you to configure execution instructions in entry or point to a local executable script file, similar to Husky.

• Scripts are tightly coupled to and distributed with the code repository.
• The state required by Hooks only exists in build artifacts of code repositories (such as virtualenv for application PyLint).
• Linter’s official repository does not provide pre-commit metadata.

Local hooks can use the language that supports Additional_dependencies or docker_image/fail/pygrep/script/system.

The repO is a local repository
- repo: local
  hooks:
  - id: pylint
    name: pylint
    entry: pylint
    language: system
    types: [python]
  - id: changelogs-rst
    name: changelogs must be rst
    entry: changelog filenames must end in .rst
    language: fail # Fail is a light language for disabling files by filename
    files: 'changelog/.*(? 
      
Copy the code

Custom local scripts

As mentioned at the beginning of this article, we want to provide a way for developers to create their own hooks, but commit them to a common code base. I looked through the official documentation and could not find the relevant function points. But with the local repo functionality above we can develop functionality that meets this requirement.

Because the Local repo allows Entry to execute local files, you can simply define an executable file for each stage. In the following configuration, a.git_hooks directory is created under the project directory to hold the developer’s own scripts. (Note that not all stages are defined here, only the stages supported by pre-commit install -t are.)

- repo: local hooks: - id: commit-msg name: commit-msg (local)    entry: .git_hooks/commit-msg language: script stages: [commit-msg]    # verbose: true - id: post-checkout name: post-checkout (local) entry: .git_hooks/post-checkout language: script stages: [post-checkout] # verbose: true - id: post-commit name: post-commit (local) entry: .git_hooks/post-commit language: script stages: [post-commit] # verbose: true - id: post-merge name: post-merge (local) entry: .git_hooks/post-merge language: script stages: [post-merge] # verbose: true - id: pre-commit name: pre-commit (local) entry: .git_hooks/pre-commit language: script stages: [commit] # verbose: true - id: pre-merge-commit name: pre-merge-commit (local) entry: .git_hooks/pre-merge-commit language: script stages: [merge-commit] # verbose: true - id: pre-push name: pre-push (local) entry: .git_hooks/pre-push language: script stages: [push] # verbose: true - id: prepare-commit-msg name: prepare-commit-msg (local) entry: .git_hooks/prepare-commit-msg language: script stages: [prepare-commit-msg] # verbose: true
Copy the code

Follow the principle of automating what you can. A script to automatically create all of the above stage files is provided (it fails if the script file specified by Entry does not exist). Install-git-reuters. sh installs stages supported by pre-commit and pre-commit, and is initialized if CUSTOMIZED=1. And add Customized local hooks to. Pre-commit -config.yaml.

install-git-hooks.sh

#! /bin/bash:<<'COMMENT'chmod +x install-git-hooks.sh./install-git-hooks.sh# intall with initializing customized hooksCUSTOMIZED=1 ./install-git-hooks.shCOMMENTSTAGES="pre-commit pre-merge-commit pre-push prepare-commit-msg commit-msg post-checkout post-commit post-merge"installPreCommit() { HAS_PRE_COMMIT=$(which pre-commit) if [ -n "$HAS_PRE_COMMIT" ]; then return fi HAS_PIP=$(which pip) if [ -z "$HAS_PIP" ]; then echo "ERROR:pip is required, please install it mantually." exit 1 fi pip install pre-commit}touchCustomizedGitHook() { mkdir .git_hooks for stage in $STAGES do STAGE_HOOK=".git_hooks/$stage" if [ -f "$STAGE_HOOK" ]; then echo "WARN:Fail to touch $STAGE_HOOK because it exists." continue fi echo -e "#! /bin/bash\n\n# general git hooks is available." > "$STAGE_HOOK" chmod +x "$STAGE_HOOK" done}preCommitInstall() { for stage in $STAGES do STAGE_HOOK=".git/hooks/$stage" if [ -f "$STAGE_HOOK" ]; then echo "WARN:Fail to install $STAGE_HOOK because it exists." continue fi pre-commit install -t "$stage" done}touchPreCommitConfigYaml() { PRE_COMMIT_CONFIG=".pre-commit-config.yaml" if [ -f "$PRE_COMMIT_CONFIG" ]; then echo "WARN: abort to init .pre-commit-config.yaml for it's existence." return 1 fi touch $PRE_COMMIT_CONFIG echo "# Unified management hooks" >> $PRE_COMMIT_CONFIG echo "# use pre-commit on Git project https://donespeak.gitlab.io/posts/210525-using-pre-commit-for-git-hooks/" >> $PRE_COMMIT_CONFIG}initPreCommitConfigYaml() { touchPreCommitConfigYaml if [ "$?" == "1" ]; then return 1 fi echo "" >> $PRE_COMMIT_CONFIG echo "repos:" >> $PRE_COMMIT_CONFIG echo " - repo: local" >> $PRE_COMMIT_CONFIG echo " hooks:" >> $PRE_COMMIT_CONFIG for stage in $STAGES do echo " - id: $stage" >> $PRE_COMMIT_CONFIG echo " name: $stage (local)" >> $PRE_COMMIT_CONFIG echo " entry: .git_hooks/$stage" >> $PRE_COMMIT_CONFIG echo " language: script" >> $PRE_COMMIT_CONFIG if [[ $stage == pre-* ]]; then stage=${stage#pre-} fi echo " stages: [$stage]" >> $PRE_COMMIT_CONFIG echo " # verbose: true" >> $PRE_COMMIT_CONFIG done}ignoreCustomizedGitHook() { CUSTOMIZED_GITHOOK_DIR=".git_hooks/" GITIGNORE_FILE=".gitignore" if [ -f "$GITIGNORE_FILE" ]; then if [ "$(grep -c "$CUSTOMIZED_GITHOOK_DIR" $GITIGNORE_FILE)" -ne '0' ]; Return fi fi echo -e "\n# Ignore. Git_hooks file in which scripts are not submitted to repository \n$CUSTOMIZED_GITHOOK_DIR\n! .git_hooks/.gitkeeper" >> $GITIGNORE_FILE}installPreCommitif [ "$CUSTOMIZED" == "1" ]; then touchCustomizedGitHook initPreCommitConfigYamlelse touchPreCommitConfigYamlfipreCommitInstallignoreCustomizedGitHook
Copy the code

Add a Makefile that provides the make install-git-hook installation instruction. The command automatically downloads the install-Git-links.sh file from the Git repository and executes it. In addition, if you run CUSTOMIZED=1 make install-git-hook, hooks for CUSTOMIZED are initialized.

Makefile

install-git-hooks: install-git-hooks.sh	chmod +x ./$< && ./$<Install git - hooks. Sh: # if you have Failed to connect to port 443 raw.githubusercontent.com: Connection refused # for DNS pollution problem, can inquire on the https://www.ipaddress.com/ domain name, and then to see # in the hosts file: https://github.com/hawtim/blog/issues/10 wget https://raw.githubusercontent.com/DoneSpeak/gromithooks/v1.0.1/install-git-hooks.sh
Copy the code

The hooks in.git_hooks can be written as scripts originally in.git/hooks, or as pre-commit hooks.

prepare-commit-msg

#! /usr/bin/env python

import argparse
from typing import Optional
from typing import Sequence

def main(argv: Optional[Sequence[str]] = None) - >int:
    parser = argparse.ArgumentParser()
    parser.add_argument('filename', nargs=The '*'.help='Filenames to check.')
    args = parser.parse_args(argv)
    # .git/COMMIT_EDITMSG
    print("commit_msg file is " + args.filename[0])

if __name__ == '__main__':
    exit(main())
Copy the code

prepare-commit-msg

#! /bin/bash

echo "commit_msg file is $1"
Copy the code

If you need to learn more about how to define git templates, you can refer to the documentation on the website.

Related articles

recommended

• This article source Code Donespeak/Gromithooks
• Define global Git Hooks and custom Git Hooks
• Associate Tapd and Commit with Git Hook

reference

• pre-commit | A framework for managing and maintaining multi-language pre-commit hooks. @pre-commit.com
• pre-commit | Supported hooks @pre-commit.com
• Yaml @github pre-commit-config.yaml @github
• Git hooks: Customize your workflow by writing Git hooks in Python
• Packaging Python Projects @python.org provides an introduction to the process from creation to release