Python provides many built-in Functions, of which 69 are listed in the latest Official Python 3 documentation.
While most of the functions are often used, such as print(), open(), and dir(), there are some functions that are not often used, but they can be useful in certain situations. Built-in functions can be “promoted,” which means they have a unique and useful use.
Therefore, mastering the use of built-in functions is a skill we should light up.
In Advanced Python: How to Convert String Constants into Variables? I’ve mentioned eval() and exec() in this article, but I don’t know much about them. To make up for it, I restudied it. This article is a super detailed study record, systematic, comprehensive and in-depth analysis of the two functions.
1. Basic uses of eval
Syntax: eval(expression, globals=None, locals=None)
It takes three arguments, of which expression is a string expression or code object that is used to perform an operation; Globals and locals are optional. The default is None.
Specifically, expression can only be a single expression and does not support complex code logic such as assignment operations, loops, and so on. (PS: A single expression does not mean “simple and harmless”, see section 4 below)
Globals specifies the global namespace for the runtime. The type is dictionary. By default, the current module’s built-in namespace is used. Locals specifies the local namespace of the runtime, which is of type dictionary and uses the value globals by default. When both default, the scope in which the eval function executes is followed. It is important to note that these two do not represent real namespaces and are only used during operations, which are then destroyed.
x = 10
def func():
y = 20
a = eval('x + y')
print('a: ', a)
b = eval('x + y', {'x': 1, 'y': 2})
print('x: ' + str(x) + ' y: ' + str(y))
print('b: ', b)
c = eval('x + y', {'x': 1, 'y': 2}, {'y': 3.'z': 4})
print('x: ' + str(x) + ' y: ' + str(y))
print('c: ', c)
func()
Copy the codeOutput results:
a: 30
x: 10 y: 20
b: 3
x: 10 y: 20
c: 4
Copy the codeAs you can see, when a namespace is specified, the variable is looked up in the corresponding namespace. Furthermore, their values do not override values in the actual namespace.
2. Basic usage of exec
Syntax: exec(object[, globals[, locals]])
Exec is a statement in Python2, and Python3 transforms it into a function, just like print. Exec () is highly similar to eval() in that the three arguments have similar meanings and effects.
The main difference is that the exec () the first parameter is not expression, but the code block, which means that two points: one is that it can’t be expression evaluation and return to go out, two is it can perform complex code logic, relatively more powerful, for example, when a new block assignment variables, the variables may survive outside the function of the namespace.
>>> x = 1
>>> y = exec('x = 1 + 1') > > >print(x)
>>> print(y)
2
None
Copy the codeAs you can see, the namespace inside and outside exec() is communicated, and variables are passed from there, unlike eval(), which requires a variable to receive the result of the function’s execution.
3, some details
Both functions are powerful in that they execute the string contents as valid code. This is a string-driven event, which is significant. In practice, however, there are many small details. Here are a few that I know of.
Common use: Turn strings into objects, such as string to list, string to dict, string to tuple, and so on.
>>> a = [[1,2], [3,4], [5,6], [7,8], [9,0]]
>>> print(eval(a))
[[1, 2], [3, 4], [5, 6], [7, 8], [9, 0]]
>>> a = "{'name': 'Python cat ', 'age': 18}"
>>> print(eval(a))
{'name': 'the Python cat'.'age'18} :# is slightly different from eval
>>> a = "My_dict = {'name': 'Python cat ', 'age': 18}"
>>> exec(a)
>>> print(my_dict)
{'name': 'the Python cat'.'age'18} :Copy the codeThe value returned by the eval() function is the result of its expression. In some cases, it will be None, such as when the expression is a print() statement or an append() operation of a list, which results in None. So the return value of eval() will also be None.
>>> result = eval('[].append(2)') > > >print(result)
None
Copy the codeThe return value of exec() will only be None, independent of the result of the execution, so there is no need to assign exec(). Statements that contain returns or yields cannot be executed outside the exec function.
>>> result = exec('1 + 1') > > >print(result)
None
Copy the codeThe globals and locals arguments in both functions act as whitelists, limiting the scope of the namespace to prevent data abuse within the scope.
The code object compiled by the conCompile () function can be used as the first argument to eval and exec. Compile () is also a magic function. My last translation, “Python SAO Operations: Dynamically Defining Functions,” demonstrates an operation that dynamically defines functions.
Paradoxical local namespaces: As mentioned earlier, variables within the exec() function can change the original namespace. However, there are exceptions.
def foo():
exec('y = 1 + 1\nprint(y)')
print(locals())
print(y)
foo()
Copy the codeAs previously understood, the expected result was that the variable y would be stored in the local variable, so both prints would result in 2, whereas the actual result was:
2
{'y': 2} Traceback (most recent call last): ... (Omitted part of the error message)print(y)
NameError: name 'y' is not defined
Copy the codeWhy is the variable y undefined when you see it in your local namespace?
The reason has to do with Python’s compiler, which first parses foo into an AST (abstract syntax tree) and then stores all the variable nodes on the stack. In this case, the exec() argument is just a string, the whole thing is constant, and it’s not executed as code, so y doesn’t exist. Until the second print() is parsed, when the variable y first appears, but because it is not fully defined, y is not stored in the local namespace.
At run time, the exec() function dynamically creates the local variable y, but because Python’s implementation mechanism is “run-time local namespaces are immutable,” that means y is never a member of the local namespace, and print() makes an error.
As for locals(), why does it not represent the true local namespace? Why cannot local namespaces be dynamically modified? Can view before I share “Python dynamic assignment of trap”, in addition, the official website bug also have to discuss this problem, check the address: bugs.python.org/issue4831
Z = locals()[‘y’]; if so, z = locals()[‘y’]; if so, z = locals()[‘y’];
def foo():
exec('y = 1 + 1')
y = locals()['y']
print(y)
foo()
# error: KeyError: 'y'
# change variable y to another variable without error
Copy the codeKeyError means that the corresponding key does not exist in the dictionary. In this example, y is declared, but the assignment cannot be completed because of a circular reference, that is, the value corresponding to the key value is an invalid value, so it cannot be read, and an error is reported.
There are four variations in this example, and I tried for a long time to explain them in a self-explanatory way, but without success. Leave it for later, when I figure it out, and write a separate article.
4. Why use eval() with caution?
Many dynamic programming languages have an eval() function that does much the same thing, but, without exception, people will tell you to avoid it.
Why use eval() with caution? Primarily for security reasons, the eval function is likely to cause code injection problems for untrusted data sources.
>>> eval("__import__('os').system('whoami')")
desktop-fa4b888\pythoncat
>>> eval("__import__('subprocess').getoutput('ls ~')")
The current path file information is omitted
Copy the codeIn the example above, my private data was exposed. Even worse, if you change the command to rm -rf ~, all files in the current directory will be deleted.
One way to limit this is to specify globals as {‘__builtins__’: None} or {‘__builtins__’: {}}.
>>> s = {'__builtins__': None}
>>> eval("__import__('os').system('whoami')", s)
TypeError: 'NoneType' object is not subscriptable
Copy the code__builtins__ contains the names in the built-in namespace, and typing dir(__builtins__) in the console will show you the names of many built-in functions, exceptions, and other attributes. By default, the globals argument to the eval function implicitly carries __builtins__, even if the globals argument is {}, so if you want to disable it, you have to explicitly specify its value.
By mapping it to None, the above example limits the built-in namespace available to eval to None, thereby limiting the expression’s ability to call built-in modules or properties.
However, this approach is not foolproof, as there are still ways to launch an attack.
A bug hunter shared an eye-opening thought on his blog. The core code is the following sentence, which you can try to execute to see what the output is.
>>> ().__class__.__bases__[0].__subclasses__()
Copy the codeFor an explanation of this code, and how to use it further, see the blog. (address: www.tuicool.com/articles/je…
There is another blog post that not only mentions the above method, but also offers a new way of thinking:
# Warning: Do not execute the following code at your own risk.
>>> eval('(lambda fc=(lambda n: [c 1="c" 2="in" 3="().__class__.__bases__[0" language="for"][/c].__subclasses__() if c.__name__ == N] [0]) : fc (" function ") (fc (" code ") (0,0,0,0, "KABOOM," (), (), (), "",", "0," "), {}) () () ', {"__builtins__":None})
Copy the codeThis line of code causes Python to crash directly. Concrete analysis in: segmentfault.com/a/119000001…
In addition to hacking, simple content can also be attacked. Writing something like the following example will exhaust the server’s computing resources in a short time.
>>> eval("2 * * 888888888", {"__builtins__":None}, {})
Copy the codeAs mentioned above, we’ve visually demonstrated the dangers of the eval() function, but even careful use by Python geeks is not guaranteed to be error-free.
In the official DumbDBM module, a security hole was discovered (in 2014) that allows attackers to launch attacks when eval() is called by forging database files. (details: bugs.python.org/issue22885)
Similarly, last month (2019.02), a core developer raised a security issue for Python 3.8 and proposed not to use eval() in logging.config, which is currently open. (details: bugs.python.org/issue36022)
All this is enough to explain why eval() should be used with caution. By the same token, the exec() function should be used with caution.
5, safe alternative usage
Given the security concerns, why create these two built-in methods? Why use them?
For the simple reason that Python is a flexible and dynamic language. Unlike static languages, dynamic languages allow code to be generated dynamically and bug fixes to already deployed projects can be made with minimal local modifications.
So what are some relatively safe ways to use them?
The ast module’s literal() is a safe alternative to eval(). Unlike eval(), which executes without checking, ast.literal() checks to see if the contents of the expression are valid. The literals it allows are as follows:
Strings, bytes, numbers, tuples, lists, dicts, sets, booleans, and None
If the content is illegal, an error will be reported:
import ast
ast.literal_eval("__import__('os').system('whoami')"ValueError: malformed node or stringCopy the codeHowever, there are drawbacks: the AST compiler has a limited stack depth, and it can crash a program if it parses too many or too complex strings.
As for exec(), there doesn’t seem to be a similar alternative, since the content it supports is much more complex and varied.
A final tip: Be aware of the differences and operational details (such as the previous local namespace content), use them carefully, limit the available namespaces, and fully validate the data source.
Related Reading:
The pitfalls of dynamic assignment in Python
Python SAO operations: Dynamically define functions
Python is home and abroad
How to convert string constants into variables?
Docs.python.org/3/library/a…
Python Cat is an official account focused on Python technology, data science and deep learning, trying to create an interesting and useful platform for learning and sharing. This number is a series of high-quality articles, such as cat star philosophy cat series, Python advanced series, good book recommendation series, excellent English recommendation and translation, etc., welcome to pay attention to oh. PS: the background reply “love learning”, get a free learning gift package.