Python craftsman: The technique of making functions return results

The preface

This is the fifth article in the “Python Craftsman” series. [View all articles in the series]

Without a doubt, functions are one of the most important concepts in the Python language. In programming, we break down big real-world problems into smaller ones and deliver the answers through function after function. Functions are the bane of repetitive code and the best defense against code complexity.

Just as most stories have an end, most functions end by returning a result. How a function returns results determines the experience of calling it. Therefore, knowing how to elegantly return a function is a prerequisite for writing a good function.

The way Python functions return

Python functions return results by calling return statements. Use return Value to return a single value, and return value1 and value2 to return multiple values at the same time.

If a function does not have any return statements, the default return value is None. In addition to returning content via a return statement, you can also “return the result” within a function by raising an Exception * (raise Exception) *.

Next, I’ll list some common programming tips related to function returns.

Programming advice
- 1. Do not return multiple types for a single function
- 2. Construct a new function using partial
- 3. Throw exceptions instead of returning results and errors
- 4. Use caution with the return value None
  - 1. As the default return value of the operation class function
  - 2. As some “expected” value that might not be there
  - 3. As the value representing the “error result” when the call fails
- 5. Use “Empty Object Mode” wisely
- 6. Use generator functions instead of return lists
- Limit the use of recursion
conclusion
The appendix

Programming advice

1. Do not return multiple types for a single function

Python is so flexible that we can easily do things that are difficult to do in other languages. For example: * Let a function return different types of results at the same time. * To implement a seemingly useful “multifunction”.

Like this:

def get_users(user_id=None):
    if user_id is None:
        return User.get(user_id)
    else:
        return User.filter(is_active=True)


# return a single user
get_users(user_id=1)
# return multiple users
get_users()
Copy the code

When we need to get a single user, we pass the user_id argument, otherwise we pass no argument and get a list of all active users. This is all done by a function called get_users. The design seems reasonable.

In the world of functions, however, it’s not a good idea to be proud of writing Swiss Army knife-shaped functions that are “multifunctional.” This is because good functions are always Single responsibility. ** Single responsibility means that a function does only one thing, with a clear purpose. Functions such as ** are also less likely to be modified in the future due to changing requirements.

Returning multiple types of functions is a violation of the “single responsibility” principle. The best function should always provide a stable return value to minimize the processing cost to the caller. ** As in the above example, we should write two separate functions get_user_by_id(user_id) and get_active_users() instead.

2. Construct a new function using partial

Imagine A scenario where you have A function A that takes A lot of arguments in your code, which is very applicable. The other function B does its work entirely by calling A, which is A kind of shortcut.

For example, in this case, the double function evaluates exactly as multiply:

def multiply(x, y):
    return x * y


def double(value):
    Return the result of another function call
    return multiply(2, value)
Copy the code

For this scenario, we can simplify it by using the partial() function in the FuncTools module.

Partial (func, *args, **kwargs) constructs a new function based on the passed function with variable (position/keyword) arguments. All calls to the new function are proxied to the original function after merging the current call parameters with the construction parameters.

With partial, the double function definition above can be modified to a single-line expression, which is cleaner and more straightforward.

import functools

double = functools.partial(multiply, 2)
Copy the code

Recommended reading: Official documentation for partial functions

3. Throw exceptions instead of returning results and errors

As I mentioned earlier, functions in Python can return multiple values. With this capability, we can write a special class of functions: functions that return both a result and an error message.

def create_item(name):
    if len(name) > MAX_LENGTH_OF_NAME:
        return None.'name of item is too long'
    if len(CURRENT_ITEMS) > MAX_ITEMS_QUOTA:
        return None.'items is full'
    return Item(name=name), ' '


def create_from_input(a):
    name = input()
    item, err_msg = create_item(name)
    if err_msg:
        print(f'create item failed: {err_msg}')
    else:
        print(f'item<{name}> created')
Copy the code

In the example, the create_item function creates a new Item object. At the same time, in order to provide the caller with error details in the event of an error, it takes advantage of the multi-return value feature, returning the error message as a second result.

At first glance, this approach seems natural. This is especially true for those who have experience programming the Go language. But in the Python world, this is not the best way to solve such problems. This increases the cost of error handling for the caller, especially if many functions follow the specification and there are multiple calls.

Python has a good * Exception * mechanism, and to some extent encourages us to use exceptions (the official documentation on EAFP). Therefore, it is more natural to use exceptions for error flow handling.

After introducing custom exceptions, the above code could be rewritten to look like this:

class CreateItemError(Exception):
    "" exception thrown when Item creation fails ""

def create_item(name):
    """ Creates a new Item: Raises CreateItemError if it cannot be created ""
    if len(name) > MAX_LENGTH_OF_NAME:
        raise CreateItemError('name of item is too long')
    if len(CURRENT_ITEMS) > MAX_ITEMS_QUOTA:
        raise CreateItemError('items is full')
    return Item(name=name)


def create_for_input(a):
    name = input()
    try:
        item = create_item(name)
    except CreateItemError as e:
        print(f'create item failed: {err_msg}')
    else:
        print(f'item<{name}> created')

Copy the code

When “throw exception” is used instead of “return (result, error message)”, the entire error process at first glance doesn’t look very different, but in fact there are a lot of differences. Some details:

The new version of the function has a more stable return value type, which will always just returnItemType or throws an exception
While I encourage the use of exceptions here, “exceptions” inevitably raise eyebrows, so it’s best to specify the types of exceptions that can be thrown in the function documentation
Exceptions, unlike return values, are reported up the call stack until they are caught. socreate_itemThe first-level caller can simply omit exception handling and hand it over to the upper level. This feature gives us more flexibility, but also more risk.

Hint: How to handle errors in a programming language is still a subject of debate. For example, the multiple return value approach, which is not recommended above, is the core error handling mechanism of the exception-free Go language. In addition, even the exception mechanism itself varies from programming language to programming language.

Abnormality, or non-abnormality, is the result of multiple choices made by the language designer, and there is no absolute superiority or superiority in most cases. However, in Python alone, using exceptions to express errors is far more consistent with the Python philosophy and should be respected.

4. Use caution with the return value None

The value None is usually used to indicate “something that should be there but is missing” **, which is unique in Python. Many programming languages have designs similar to None, such as null in JavaScript, nil in Go, etc. None is often used as a function return value because of its unique nihilism.

When we use None as a function return value, there are usually three cases.

1. As the default return value of the operation class function

When an operation-class function does not require any return value, None is usually returned. None is also the default return value for functions without any return statements.

For such functions, None is fine, as are list.append() and os.chdir() in the standard library.

2. As some “expected” value that might not be there

There are functions whose purpose is usually to try to do something. Depending on the situation, there may or may not be an outcome. For the caller, “no result” is completely predictable. It is also reasonable for such functions to use None as the return value when there is no result.

In the Python standard library, the re.search and re.match functions in the regular expression module re fall into this category, returning the Re. match object if a match can be found, and None if not.

3. As the value representing the “error result” when the call fails

Sometimes, None is often used as the default return value when a function call fails, such as the following function:

def create_user_from_name(username):
    "" create a User instance by username ""
    if validate_username(username):
        return User.from_username(username)
    else:
        return None


user = create_user_from_name(username)
if user:
    user.do_something()
Copy the code

Create_user_from_name returns None when username is invalid. But in this case, that’s not really a good idea.

But you might think that this function makes perfect sense, and you might even think that it’s very similar to our previous “no result” use. So how do you distinguish between these two different situations? The key is whether there is an “expected” implication between the function signature (name and argument) and the return value of None.

Let me explain. Every time you tell a function to return None, read the function name carefully and ask yourself one question: If I am the user of the function, is “getting no results” part of the meaning of the function’s name?

Use these two functions for example:

re.search()From the function name,search, which means to go from the target stringsearchThe match result, and the search behavior, always may or may not have a result, so this function is suitable to return None
create_user_from_name()In terms of the function name, it means to build the user based on a name, and not read oneMay return, may not returnThe meaning of. So it’s not appropriate to return None

For functions whose names cannot be read as None, there are two ways to modify them. First, if you insist on returning None, change the name of the function. For example, you can rename create_user_from_name() to create_user_or_none().

The second method is much more common: raise an Exception * (raise Exception) instead of None. Because if failing to return a normal result is not part of the meaning of the function, it means that the function has “unexpected conditions” *, and that’s where Exceptions come in.

An example using exception overwrites:

Class UnableToCreateUser(Exception): """ def create_user_from_name(username): "" Create a User instance from the User name ":raises UnableToCreateUser when the User cannot be created "" if validate_username(username): return User.from_username(username) else: raise UnableToCreateUser(f'unable to create user from {username}') try: user = create_user_from_name(username) except UnableToCreateUser: # Error handling else: user.do_something()Copy the code

Throwing an exception has an additional advantage over the None return value, in addition to the characteristics we mentioned in the last scenario: you can provide the reason for the unexpected result in the exception message in a way that just returning A None value cannot.

5. Use “Empty Object Mode” wisely

I mentioned earlier that a function can return an error result with a Value of None or an exception, but both methods have a common disadvantage. Where a function return value is required, an if or try/except defense statement must be added to determine if the result is normal.

Let’s look at a full working example:

import decimal


class CreateAccountError(Exception):
    """Unable to create a account error"""


class Account:
    """ A virtual bank account """

    def __init__(self, username, balance):
        self.username = username
        self.balance = balance

    @classmethod
    def from_string(cls, s):
        """ Initializes an account from a string.
        try:
            username, balance = s.split()
            balance = decimal.Decimal(float(balance))
        except ValueError:
            raise CreateAccountError('input must follow pattern "{ACCOUNT_NAME} {BALANCE}"')

        if balance < 0:
            raise CreateAccountError('balance can not be negative')
        return cls(username=username, balance=balance)


def caculate_total_balance(accounts_data):
    Calculate the total balance of all accounts.
    result = 0
    for account_string in accounts_data:
        try:
            user = Account.from_string(account_string)
        except CreateAccountError:
            pass
        else:
            result += user.balance
    return result


accounts_data = [
    'piglei 96.5'.'cotton 21'.'invalid_data'.'roland $invalid_balance'.'alfred -3',
]

print(caculate_total_balance(accounts_data))
Copy the code

In this example, whenever we call account.from_string, we must use try/except to catch possible exceptions. If the function needs to be called many times in your project, this part of the work can become tedious. In this case, Null object pattern can be used to improve the control flow.

Martin Fowler describes this pattern in detail in a chapter in his classic book refactoring. Simply put, this is to use a “null type” that conforms to the normal result interface instead of a null value return/throw exception to reduce the cost of processing the result for the caller.

With the introduction of the “empty object pattern”, the above example could be modified to look like this:

class Account:
    # def __init__ has been omitted... .
    
    @classmethod
    def from_string(cls, s):
        Initialize an Account from a string :returns Account Object if the input is valid, NullAccount otherwise.
        try:
            username, balance = s.split()
            balance = decimal.Decimal(float(balance))
        except ValueError:
            return NullAccount()

        if balance < 0:
            return NullAccount()
        return cls(username=username, balance=balance)


class NullAccount:
    username = ' '
    balance = 0

    @classmethod
    def from_string(cls, s):
        raise NotImplementedError
Copy the code

In the new version of the code, I’ve defined a new NullAccount type to return as the error result of a from_string failure. The biggest change after this modification is in the caculate_total_balance section:

def caculate_total_balance(accounts_data):
    Calculate the total balance of all accounts.
    return sum(Account.from_string(s).balance for s in accounts_data)
Copy the code

Instead of explicitly using a try statement to handle errors, callers can assume that the account.from_string function always returns a valid Account object, greatly simplifying the entire calculation logic.

Hint: In the Python world, “null object mode” is not uncommon. AnonymousUser of the well-known Django framework is a typical null object.

6. Use generator functions instead of return lists

It is particularly common to return lists in functions. Normally, we would initialize a list results = [], then fill it with the results.append(item) function in the body of the loop, and finally return at the end of the function.

For this type of pattern, we can simplify it with generator functions. To put it crudely, we use yield Item instead of append. Functions that use generators are generally more concise and general-purpose.

def foo_func(items):
    for item in items:
        #... After processing an item, use yield to return it
        yield item
Copy the code

I examined this pattern in detail in the fourth article in the series, “The Gateway to Containers.” For more details, visit the article and search for “Write Code that Scales Better.”

Limit the use of recursion

Recursion occurs when a function is called back to itself. Recursion is a very useful programming technique in certain scenarios, but the bad news is that the Python language has very limited support for recursion.

This “limited support” manifests itself in many ways. First, the Python language does not support tail-recursive optimization. Python also has a strict limit on the maximum number of recursion levels.

So my advice: write recursion as little as possible. If you want to solve a problem recursively, consider whether it is convenient to use a loop instead. If the answer is yes, use a loop to rewrite it. If you have to use recursion, consider the following points:

Whether the input data size of the function is stable, and whether it must not exceedsys.getrecursionlimit()Maximum number of layers specified
Whether the number of recursive layers can be reduced by using a cache utility function like functools.lru_cache

conclusion

In this article, I’ve simulated a few scenarios related to Python function returns and provided my optimization suggestions for each scenario. Finally, to summarize the main points:

A function only does one thing by having a stable return value
usefunctools.partialDefine shortcut functions
Throwing an exception is also a way to return a result instead of returning an error message
Whether a function is appropriate to return None depends on the “meaning” of the function signature
Using “empty object mode” simplifies error handling logic for callers
Use generator functions, and replace recursion with loops

After reading the article, do you have anything to tease? Let me know in the comments or on Github Issues.

The appendix

Dominik Scythe on Unsplash

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Python craftsman: The technique of making functions return results

The preface

The way Python functions return

Table of contents

Programming advice

1. Do not return multiple types for a single function

2. Construct a new function using partial

3. Throw exceptions instead of returning results and errors

4. Use caution with the return value None

1. As the default return value of the operation class function

2. As some “expected” value that might not be there

3. As the value representing the “error result” when the call fails

5. Use “Empty Object Mode” wisely

6. Use generator functions instead of return lists

Limit the use of recursion

conclusion

The appendix

Python craftsman: The technique of making functions return results

The preface

The way Python functions return

Table of contents

Programming advice

1. Do not return multiple types for a single function

2. Construct a new function using partial

3. Throw exceptions instead of returning results and errors

4. Use caution with the return value None

1. As the default return value of the operation class function

2. As some “expected” value that might not be there

3. As the value representing the “error result” when the call fails

5. Use “Empty Object Mode” wisely

6. Use generator functions instead of return lists

Limit the use of recursion

conclusion

The appendix

Related Posts

N tips to write more efficient Dockerfile | cloud effect engineers refer to north

Java to obtain the wechat official account added events

Concurrent programming