The original video address: the Tools to Manage Large Python Codebases | Fabio Fleitas @ PyBay2018

More related articles are also available:

  • Ask HN: How do you handle large Python projects?

Tool in

  • pipenv
  • flake8
  • coverage.py
  • python-dotenv
  • bandit
  • safety
  • pre-commit
  • services

Environmental management: Piper V

Github address: github.com/pypa/pipenv

There are many articles about Pipenv, so I won’t go into details. Here is an excellent article:

  • Pipenv makes managing your Python development environment more elegant

style & linting : flake8

Github address: github.com/PyCQA/flake…

Address: document flake8.pycqa.org/en/latest/i…

Basic usage:

Check the whole project:

flake8 path/to/your_project/
Copy the code

Checking a single file:

flake8 path/to/your_file.py 
Copy the code

Check for a specific type of flag, for example IF I want to check which try except does not specify a specific Exception type:

flake8 --select E722 . | more
Copy the code

Detailed what flag can view here: flake8.pycqa.org/en/latest/u… .

Flake8 configuration file

Document in: flake8.pycqa.org/en/latest/u…

You can add the tox.ini, setup.cfg, and. Flake8 files in the project root directory.

[flake8]
ignore = D203
exclude =
    # No need to traverse our git directory
    .git,
    # There's no value in checking cache directories
    __pycache__,
    # The conf file is mostly autogenerated, ignore it
    docs/source/conf.py,
    # The old directory contains Flake8 2.0
    old,
    # This contains our built documentation
    build,
    # This contains builds of flake8 that we don't want to check
    dist
max-complexity = 10
Copy the code

Is equivalent to:

flake8 --ignore D203 \
    --exclude .git,__pycache__,docs/source/conf.py,old,build,dist \
    --max-complexity 10
Copy the code

pre-commit hooks

Pre-commit documents: pre-commit.com/#pre-commit…

Example: Add the following configuration to. Pre-commit-config.yaml:

-   repo: https://gitlab.com/pycqa/flake8
    rev: ' '  # pick a git hash / tag to point to
    hooks:
    -   id: flake8
Copy the code

code coverage : coverage.py

Github address: github.com/nedbat/cove…

Code coverage:

Code coverage is a measurement of how many lines/blocks/arcs of your code are executed while the automated tests are running. It comes from this.

Which lines and blocks of code are actually executed and which are not when you run an automated test. It is an important indicator to measure the comprehensiveness of UNITTest.

PIP install Coverage PIP install coverage

For example, in Django:

coverage run --source='. ' manage.py test
coverage html
Copy the code

Htmldov / : htmlcov/ : http-server: htmldov/ : http-server: htmldov/

Click on a file to see which lines are executed at./manage.py test:

If you just want to read a coverage report, you can use a coverage report.

Distinguish online and offline environment configuration: python-dotenv

Github address: github.com/theskumar/p…

Simply put, put the configuration information in a separate configuration file (such as.env), decoupled from the code.

(5) I talked about this in detail in my last article, only with python-ple, but the idea is the same. I’m not going to do that here.

Security vulnerability detection: Bandit

Github address: github.com/PyCQA/bandi…

Basic usage (-r checks all files in the current directory) :

bandit -r . | more
Copy the code

The ability to locate the specific line, and the most document links. As here detected pseudo – random generators, it gives the corresponding document address: bandit. Readthedocs. IO/en/latest/b…

There is a summary at the end:

Bandit also has a specific configuration file, see the official documentation.

Third party dependent vulnerability detection: safety

Github address: github.com/pyupio/safe…

Bindit checks for your own code, while Safety checks for third-party dependencies that follow.

safety check 
Copy the code

A django security vulnerability with ID 36769 has been detected.

The details of this number can be viewed in this file. CVE numbers can be found on this website.

Corresponding django official instructions located at: www.djangoproject.com/weblog/2019… . Refers to the django. Views. Defaults. Page_not_found () this method, an attacker could be in 404 pages into specific content.

For example, a default 404 page would look like this: path/Hello would be displayed.

If the attacker changed /hello:

It is possible to induce users to click on phishing sites.

Comment on the Django vulnerability CVE-2019-3498

pre-commit

Pre-commit is a set of hook bash scripts that you can run when performing a specific Git operation. For example, you can run flake8, Bandit, and Safety before committing or pushing.

Other services

CI

Error Tracking

  • Github.com/getsentry/s…

If you love computer science and basic logic like me, welcome to follow my wechat official account: