A list,

The Pydantic library is python’s library for checking and managing data interface definitions.

Pydantic enforces type hints at run time and provides friendly errors when data is invalid.

It has the following advantages:

  1. Works perfectly with IDE/ Linter, no new schema needs to be learned, just use type annotations to define instances of classes
  2. Versatile, BaseSettings can both validate request data and read system Settings from environment variables
  3. fast
  4. Can verify complex structures
  5. Extensible and usablevalidatorDecorator decorates methods on models to extend validation
  6. Data class integration, in addition toBaseModel, Pydantic also provides onedataclassDecorator, which creates plain Python data classes with input data parsing and validation.

Second, the installation

pip install pydantic
Copy the code

To test if PyDantic is compiled, run:

import pydantic
print('compiled:', pydantic.compiled)
Copy the code

The configuration can be obtained using the dotenv file. Python-dotenv must be installed

pip install pydantic[dotenv]
Copy the code

3. Common models

Objects in PyDantic are defined through models, which you can think of as types in a type language.

1. BaseModel basic model

from pydantic import BaseModel

class User(BaseModel):
    id: int
    name = 'Jane Doe'
Copy the code

The above example defines a User model, derived from BaseModel, with two fields, id being an integer and required, and name being a string with a default value and not required

Instantiation use:

user = User(id='123')
Copy the code

Instantiation will perform all parsing and validation, and a ValidationError will be raised if there is an error.

The model has the following properties:

  • Dict () a dictionary of model fields and values

  • Json () JSON string representing dict()

  • Copy () copy of the model (shallow copy by default)

  • Parse_obj () parses data using dict

  • Parse_raw takes STR or bytes and parses it into JSON, then passes the result to parse_obj

  • Parse_file file path that reads the file and passes the content to parse_RAW. If content_type is omitted, it is inferred from the file extension

  • From_orm () creates the model from the ORM object

  • Schema () returns a dictionary of schemas

  • Schema_json () returns a JSON string representation of the dictionary

  • Construct () allows models to be created without validation

  • __fields_set__ Set of field names set when the model instance is initialized

  • __fields__ Dictionary of model fields

  • The configuration class for the __config__ model

2. Recursive model

You can define more complex data structures using the model itself as a type in the annotations.

from typing import List
from pydantic import BaseModel


class Foo(BaseModel):
    count: int
    size: float = None


class Bar(BaseModel):
    apple = 'x'
    banana = 'y'


class Spam(BaseModel):
    foo: Foo
    bars: List[Bar]
Copy the code

GenericModel GenericModel

Use typing. TypeVar instance as a parameter, pass to typing, Generic, and then inherited pydantic. Generics. GenericModel model used in:

from typing import Generic, TypeVar, Optional, List

from pydantic import BaseModel, validator, ValidationError
from pydantic.generics import GenericModel

DataT = TypeVar('DataT')


class Error(BaseModel):
    code: int
    message: str


class DataModel(BaseModel):
    numbers: List[int]
    people: List[str]


class Response(GenericModel, Generic[DataT]):
    data: Optional[DataT]
    error: Optional[Error]

    @validator('error', always=True)
    def check_consistency(cls, v, values):
        if v is not None and values['data'] is not None:
            raise ValueError('must not provide both data and error')
        if v is None and values.get('data') is None:
            raise ValueError('must provide data or error')
        return v


data = DataModel(numbers=[1, 2, 3], people=[])
error = Error(code=404, message='Not found')

print(Response[int](data=1))
#> data=1 error=None
print(Response[str](data='value'))
#> data='value' error=None
print(Response[str](data='value').dict())
#> {'data': 'value', 'error': None}
print(Response[DataModel](data=data).dict())
"""
{
    'data': {'numbers': [1, 2, 3], 'people': []},
    'error': None,
}
"""
print(Response[DataModel](error=error).dict())
"""
{
    'data': None,
    'error': {'code': 404, 'message': 'Not found'},
}
"""
try:
    Response[int](data='value')
except ValidationError as e:
    print(e)
    """
    2 validation errors for Response[int]
    data
      value is not a valid integer (type=type_error.integer)
    error
      must provide data or error (type=value_error)
    """
Copy the code

4, create_model dynamic model

In some cases, the structure of the model is not known until runtime. Pydantic provides create_model to allow dynamic model creation.

from pydantic import BaseModel, create_model DynamicFoobarModel = create_model('DynamicFoobarModel', foo=(str, ...) , bar=123)Copy the code

Four, common types

  • None, type(None), or Literal[None] only values of None are allowed

  • Bool Indicates the Boolean type

  • Int Integer type

  • Float Indicates the floating point number type

  • STR string type

  • Bytes Indicates the byte type.

  • Lists allow list, the tuple, set, frozenset, deque, or generator and converted into a list

  • Tuple allow list, tuple, set, frozenset, deque, or a set of generator and converted into RMB

  • Dict dictionary type

  • Set to allow list, tuple, set, frozenset, deque, converting or generator and collection;

  • Frozenset allow list, tuple, set, frozenset, deque, or generator set and cast to freeze

  • Deque allow list, tuple, set, frozenset, deque, or generator and forced into a deque

  • Datetime date and datetime, time, timedelta date type, etc

  • Deque, Dict, FrozenSet, List, Optional, Sequence, Set, Tuple, Union, Callable, Pattern, etc

  • FilePath: indicates the FilePath

  • DirectoryPath DirectoryPath

  • EmailStr Email address

  • NameEmail Valid E-mail address or format

  • PyObject takes a string and loads Python objects that can be imported in the dotted path;

  • Color Color type

  • AnyUrl AnyUrl

  • SecretStr, SecretBytes sensitive information, will be formatted as ‘**********’ or ”

  • Json type

  • PaymentCardNumber Type of the payment card

  • Constraint type. You can restrict values of many common types using con* type functions

    • conlist
  1. item_type: Type[T]: Indicates the type of the list item
  2. min_items: int = None: Minimum number of items in the list
  3. max_items: int = None: The maximum number of items in the list
  • conset
  1. item_type: Type[T]: Sets the project type
  2. min_items: int = None: The minimum number of items in the collection
  3. max_items: int = None: The maximum number of items in the collection
  • conint
  1. strict: bool = False: Control type force
  2. gt: int = None: Forces the integer to be greater than the set value
  3. ge: int = None: Forces an integer to be greater than or equal to a set value
  4. lt: int = None: Forces the integer to be less than the specified value
  5. le: int = None: Forces an integer to be less than or equal to a set value
  6. multiple_of: int = None: Forces an integer to be a multiple of the set value
  • confloat
  1. strict: bool = False: Control type force
  2. gt: float = None: forces a floating point number to be greater than the set value
  3. ge: float = None: forces float to be greater than or equal to the set value
  4. lt: float = None: Forces a floating point number to be less than the set value
  5. le: float = None: Forces float to be less than or equal to the set value
  6. multiple_of: float = None: forces float to be a multiple of the set value
  • condecimal
  1. gt: Decimal = None: Specifies that the value is greater than the set value in decimal notation
  2. ge: Decimal = None: Specifies a decimal value greater than or equal to the set value
  3. lt: Decimal = None: Specifies that the decimal value is less than the set value
  4. le: Decimal = None: Specifies that the decimal value is less than or equal to the set value
  5. max_digits: int = None: The largest number of digits in the decimal point. It does not include zeros before the decimal point or decimal zeros following it
  6. decimal_places: int = None: Maximum number of decimal places allowed. It does not include trailing decimal zeros
  7. multiple_of: Decimal = None: Specifies the decimal multiple of the set value
  • constr
  1. strip_whitespace: bool = False: Deletes Spaces at the beginning and end
  2. to_lower: bool = False: Converts all characters to lowercase
  3. strict: bool = False: Control type force
  4. min_length: int = None: Minimum length of the string
  5. max_length: int = None: Maximum length of a string
  6. curtail_length: int = None: Shrinks the string length to the specified value when the string length exceeds the specified value
  7. regex: str = None: regular expressions to validate strings
  • conbytes
  1. strip_whitespace: bool = False: Deletes Spaces at the beginning and end
  2. to_lower: bool = False: Converts all characters to lowercase
  3. min_length: int = None: Minimum length of a byte string
  4. max_length: int = None: Maximum length of a byte string

  • Strict types, you can use StrictStr, StrictBytes, StrictInt, StrictFloat, and StrictBool types to prevent mandatory compatibility types

Five, the validator

Use the Validator decorator to implement custom validations and complex relationships between objects.

from pydantic import BaseModel, ValidationError, validator class UserModel(BaseModel): name: str username: str password1: str password2: str @validator('name') def name_must_contain_space(cls, v): if ' ' not in v: raise ValueError('must contain a space') return v.title() @validator('password2') def passwords_match(cls, v, values, **kwargs): if 'password1' in values and v ! = values['password1']: raise ValueError('passwords do not match') return v @validator('username') def username_alphanumeric(cls, v): assert v.isalnum(), 'must be alphanumeric' return v user = UserModel( name='samuel colvin', username='scolvin', password1='zxcvbn', password2='zxcvbn', ) print(user) #> name='Samuel Colvin' username='scolvin' password1='zxcvbn' password2='zxcvbn' try: UserModel( name='samuel', username='scolvin', password1='zxcvbn', password2='zxcvbn2', ) except ValidationError as e: print(e) """ 2 validation errors for UserModel name must contain a space (type=value_error) password2 passwords do not match (type=value_error) """Copy the code

Some notes about validators:

  • Validators are “class methods,” so the first parameter value they receive isUserModelClass instead ofUserModel
  • The second argument is always the value of the field to validate, and you can name it whatever you want
  • A single validator can be applied to multiple fields by passing multiple field names, or a single validator can be invoked on all fields by passing special valuesThe '*'
  • Keyword parameterpreWill cause the validator to be called before any other validations
  • througheach_item=TrueWill cause the validator to be applied to a separate value (e.gList.Dict.SetEtc.), rather than the whole object
from typing import List from pydantic import BaseModel, ValidationError, validator class ParentModel(BaseModel): names: List[str] class ChildModel(ParentModel): @validator('names', each_item=True) def check_names_not_empty(cls, v): assert v ! = ' ' 'Empty strings are not allowed.' return v # This will NOT raise a ValidationError because the validator was not called try: child = ChildModel(names=['Alice', 'Bob', 'Eve', '']) except ValidationError as e: print(e) else: print('No ValidationError caught.') #> No ValidationError caught. class ChildModel2(ParentModel): @validator('names') def check_names_not_empty(cls, v): for name in v: assert name ! = '', 'Empty strings are not allowed.' return v try: child = ChildModel2(names=['Alice', 'Bob', 'Eve', '']) except ValidationError as e: print(e) """ 1 validation error for ChildModel2 names Empty strings are not allowed. (type=assertion_error) """Copy the code
  • The keyword argument always causes always validation, and for performance reasons, by default, the validator is not invoked for a field when no value is provided. However, there are situations where it may be useful or necessary to always call the validator, such as setting dynamic defaults.
  • Allow_reuse can use the same validator on multiple fields/models
from pydantic import BaseModel, validator


def normalize(name: str) -> str:
    return ' '.join((word.capitalize()) for word in name.split(' '))


class Producer(BaseModel):
    name: str

    # validators
    _normalize_name = validator('name', allow_reuse=True)(normalize)


class Consumer(BaseModel):
    name: str

    # validators
    _normalize_name = validator('name', allow_reuse=True)(normalize)
Copy the code

Six, configuration,

If you create a model inherited from BaseSettings, the model initializer will try to determine the values of any fields not passed as keyword arguments by reading from the environment. (If no matching environment variable is set, the default value is still used.)

This makes it easy to:

  • Create well-defined, type-suggestive application configuration classes
  • Automatically reads configuration changes from environment variables
  • Manually override specific Settings in initializers where needed (for example, in unit tests)
from typing import Set from pydantic import ( BaseModel, BaseSettings, PyObject, RedisDsn, PostgresDsn, Field, ) class SubModel(BaseModel): foo = 'bar' apple = 1 class Settings(BaseSettings): auth_key: str api_key: str = Field(... , env='my_api_key') redis_dsn: RedisDsn = 'redis://user:pass@localhost:6379/1' pg_dsn: PostgresDsn = 'postgres://user:pass@localhost:5432/foobar' special_function: PyObject = 'math.cos' # to override domains: # export my_prefix_domains='["foo.com", "bar.com"]' domains: Set[str] = set() # to override more_settings: # export my_prefix_more_settings='{"foo": "x", "apple": 1}' more_settings: SubModel = SubModel() class Config: env_prefix = 'my_prefix_' # defaults to no prefix, i.e. "" fields = { 'auth_key': { 'env': 'my_auth_key', }, 'redis_dsn': { 'env': ['service_redis_dsn', 'redis_url'] } } print(Settings().dict()) """ { 'auth_key': 'xxx', 'api_key': 'xxx', 'redis_dsn': RedisDsn('redis://user:pass@localhost:6379/1', scheme='redis', user='user', password='pass', host='localhost', host_type='int_domain', port='6379', path='/1'), 'pg_dsn': PostgresDsn('postgres://user:pass@localhost:5432/foobar', scheme='postgres', user='user', password='pass', host='localhost', host_type='int_domain', port='5432', path='/foobar'), 'special_function': <built-in function cos>, 'domains': set(), 'more_settings': {'foo': 'bar', 'apple': 1}, } """Copy the code

Support Dotenv file setting variable, Pydantic has two ways to load it:

class Settings(BaseSettings):
    ...

    class Config:
        env_file = '.env'
        env_file_encoding = 'utf-8'
Copy the code

or

settings=Settings(_env_file='prod.env',_env_file_encoding='utf-8')
Copy the code

Even if a Dotenv file is used, PyDantic will still read environment variables, which will always take precedence over values loaded from the Dotenv file.

Pydantic supports setting sensitive information files, which can also be loaded in two ways:

class Settings(BaseSettings):
    ...
    database_password: str

    class Config:
        secrets_dir = '/var/run'
Copy the code

Or:

settings = Settings(_secrets_dir='/var/run')
Copy the code

Even if you use the Secrets directory, PyDantic will still read environment variables from dotenv files or environment, and dotenv files and environment variables will always take precedence over values loaded from the Secrets directory.

7. Use with MYPY

Pydantic comes with a Mypy plug-in that adds a number of important Pydantic specific features to Mypy to improve its ability to type check code.

For example, the following script:

from datetime import datetime
from typing import List, Optional
from pydantic import BaseModel, NoneStr


class Model(BaseModel):
    age: int
    first_name = 'John'
    last_name: NoneStr = None
    signup_ts: Optional[datetime] = None
    list_of_ints: List[int]


m = Model(age=42, list_of_ints=[1, '2', b'3'])
print(m.middle_name)  # not a model field!
Model()  # will raise a validation error for age and list_of_ints
Copy the code

In the absence of any special configuration, mypy catches one of these errors:

13: error: "Model" has no attribute "middle_name"
Copy the code

When the plug-in is enabled, it captures both:

13: error: "Model" has no attribute "middle_name"
16: error: Missing named argument "age" for "Model"
16: error: Missing named argument "list_of_ints" for "Model"
Copy the code

To enable the plug-in, simply add Pydantic.mypy to the list of plug-ins in the mypy configuration file:

[mypy]
plugins = pydantic.mypy
Copy the code

To change the value of the plug-in Settings, create a section named [Pydantic-Mypy] in the mypy configuration file and add key-value pairs for the Settings to override:

[mypy]
plugins = pydantic.mypy

follow_imports = silent
warn_redundant_casts = True
warn_unused_ignores = True
disallow_any_generics = True
check_untyped_defs = True
no_implicit_reexport = True

# for strict mypy: (this is the tricky one :-))
disallow_untyped_defs = True

[pydantic-mypy]
init_forbid_extra = True
init_typed = True
warn_required_dynamic_aliases = True
warn_untyped_fields = True
Copy the code