This is the 23rd day of my participation in the August More Text Challenge

In our program development, we often need to request some external interface resources, and we can not guarantee that every request will be successful, so these code fragments involving network requests need to add retry mechanism. Here’s a look at the retry method in Python.

Cyclic addition judgment

The simplest way to retry is to put a loop around the code snippet that needs to be retried, catch exceptions in the program, exit the loop on success, and repeat the code on failure, for example:

import requests


def req_with_retry(url) :
    retry_max = 10  The maximum retry times is 10
    for i in range(1, retry_max+1) :try:
            print("Request {} th".format(i))
            The ConnectTimeout exception will not be thrown
            res = requests.get(url, timeout=1)

            data = res.json()
            print("Request successful:", data)
            break
        except requests.exceptions.ConnectTimeout as e:
            continue


Request a url that does not exist
req_with_retry("https://www.hahaha.cn/haha")
Copy the code

Execution Result:



Because a non-existent url was requested, I kept retrying until the maximum number of times was 10. But this is intrusive code, and it’s not pretty to add loop judgment to the business logic. Don’t worry, there’s a better way.

retrying

Retrying is a third party library in Python that provides a retry decorator function. The decorator function is retried if it fails. By default, it is retried whenever an error is reported until it succeeds. You can install it using PIP Install Retrying. For example, in the following code, we use the way of generating the size of random number to simulate the success and failure of business. As long as the generated random number is greater than 2, it will be regarded as failure and retry until the generated random number is less than 2:

import random
from retrying import retry


@retry
def random_with_retry() :
    if random.randint(0.10) > 2:
        print("Greater than 2, retry...")
        raise Exception("Greater than 2")
    print("Less than 2, success!")


random_with_retry()
Copy the code

The running results are as follows:

Retry also accepts a number of arguments, which are optional in the Retrying initialization function in the source code:

  • Stop_max_attempt_number: maximum number of retries. If this number is exceeded, the retries are stopped
  • Stop_max_delay: the maximum delay (the total time to retry this method), after which the system stops
  • Wait_fixed: indicates the wait time between two Retrying attempts
  • Wait_random_min and wait_random_max: Generate the wait time between retrying in a random way
  • Wait_incrementing_start and wait_INCREmenting_increment: increases by a fixed amount per call
  • Wait_exponential_multiplier and wait_exponential_max: Attempt_number * wait_exponential_multiplier, produces an index of the time to wait between retrying and 2^previous_attempt_number * wait_exponential_multiplier, Previous_attempt_number is the previous retry number, and if the value generated exceeds wait_exponential_max, the remaining values between the next two Retries are wait_exponential_max.

Of particular note is the retry_on_exception parameter, which accepts a function used as follows:

# Determine the exception
def is_MyError(exception) :
    print("Abnormal judgment", exception)
    print(isinstance(exception, (ValueError, IOError, ConnectionError)))
    return isinstance(exception, (ValueError, IOError, ConnectionError))


@retry(retry_on_exception=is_MyError)
def random_with_retry() :
    "" a random integer between 0 and 10, greater than 2, less than 2 success :return: ""
    if random.randint(0.10) > 2:
        print("Greater than 2, retry...")
        raise ValueError("Greater than 2")
    print("Less than 2, success!")


random_with_retry()
Copy the code

The retry_on_exception parameter receives a custom is_MyError function. Is_MyError determines whether ValueError, IOError, and ConnectionError are exceptions. The random_with_retry() function, if it throws an exception, goes to is_MyError() to determine whether it returns True or False, retry if True, or stop immediately and throw the exception if False.

There is also the retry_on_result parameter, which also receives a function that needs to be retried to determine which results the business function returns, similar in idea to the retry_on_exception parameter.

We can reasonably match these parameters according to our own needs to achieve the desired effect.

Finally, thank my girlfriend for her tolerance, understanding and support in work and life!