Python bindings: Call C or C++ from Python

Abstract:Are you a Python developer who has C or C++ libraries that you want to use from Python? If so, Python bindings allow you to call functions and pass data from Python to C or C++, allowing you to take advantage of both languages.

Share this article from huawei cloud community the Python bindings: from the Python call C or C + + | grow! Python! “】, the original author: Yuchuan.

Are you a Python developer who has C or C++ libraries that you want to use from Python? If so, Python bindings allow you to call functions and pass data from Python to C or C++, allowing you to take advantage of both languages. In this tutorial, you’ll see an overview of some of the tools you can use to create Python bindings.

In this tutorial, you will learn:

Why call C or C++ from Python
How do I pass data between C and Python
What tools and methods can help you create Python bindings

This tutorial is intended for intermediate Python developers. It assumes a basic knowledge of Python and an understanding of functions and data types in C or C++. You can get all of the sample code you’ll see in this tutorial by clicking the link below:

Summary of the Python Bindings

Before delving into how to call C from Python, it’s a good idea to take a moment to understand why. There are several situations where it is a good idea to create a Python binding to call a C library:

You already have a large, tested, stable library written in C++ that you want to leverage in Python. This could be a communication library or a library that talks to a particular piece of hardware. It doesn’t matter what it does.
You want to speed up a particular part of your Python code by converting critical parts to C. C not only has a faster execution time, but it also allows you to escape the limitations of GIL, if you are careful.
You want to test their system on a large scale using Python testing tools.

All of these are important reasons to learn to create Python bindings to interact with C libraries.

Note: In this tutorial, you will create Python bindings to C and C++. Most common concepts apply to both languages, so C will be used unless there is a specific difference between the two languages. Typically, each tool supports either C or C++, but not both.

Let’s get started!

Marshal data types

Waiting for! Before you start writing Python bindings, take a look at how Python and C store data and what types of problems this can cause. First, let’s define marshalling. This concept is defined by Wikipedia as follows:

The process of converting the memory representation of an object to a data format suitable for storage or transmission.

For your purposes, marshalling is what Python bindings do when they prepare data to move it from Python to C or vice versa. Python bindings need to be marshalled because Python and C store data differently. C stores data in memory in its most compact form. If you use uint8_t, it will use only 8 bits of memory in total.

In Python, on the other hand, everything is an object. This means that each integer uses several bytes in memory. How much depends on what version of Python you’re running, your operating system, and other factors. This means that your Python binding will need to convert C integers to Python integers for each integer passed across boundaries.

Other data types have similar relationships between the two languages. Let’s take a look:

Integer stores count numbers. Python stores integers with arbitrary precision, which means you can store very, very large numbers. C Specifies the exact size of the integer. You need to pay attention to the data size when moving between languages to prevent Python integer values from overflowing C integer variables.
Floating-point numbers are numbers with decimal places. Python can store much larger (and smaller) floating-point numbers than C. This means that you must also pay attention to these values to ensure that they remain in range.
A complex number is a number with an imaginary part. While Python has built-in complex numbers and C has complex numbers, there is no built-in method for marshalling between them. To marshal complex numbers, you need to build structs or classes in C code to manage them.
A string is a sequence of characters. As such a common data type, strings can prove quite tricky when you create Python bindings. As with other data types, Python and C store strings in completely different formats. (Unlike other data types, this is also an area where C and C++ differ, which adds to the fun!) Each of the solutions you’ll look at has a slightly different way of handling strings.
Boolean variables can have only two values. Since they are supported in C, grouping them will prove to be fairly simple.

In addition to data type conversions, there are other issues to consider when building Python bindings. Let’s keep exploring them.

Understand variable and immutable values

In addition to all these data types, you must also understand how Python objects are mutable or immutable. C has a similar concept of function arguments when it comes to passing values or references. In C, all parameters are passed by value. If you want to allow a function to change a variable in the caller, you need to pass a pointer to that variable.

You might wonder if you can bypass the immutable limit by simply passing immutable objects to C using a pointer. Python won’t give you a pointer to object unless you go to the ugly and non-portable extreme, so it won’t work. If you want to modify Python objects in C, you need to take additional steps to do so. These steps will depend on the tool you use, as shown below.

Therefore, you can add immutability to the list of items to consider when you create Python bindings. Your final stop on the grand journey of creating this listing is how to deal with the different ways Python and C deal with memory management.

Memory management

C and Python manage memory differently. In C, the developer must manage all memory allocations and ensure that they are freed once and only once. Python uses the garbage collector to handle this for you.

While each of these approaches has its advantages, it does add additional hassle to creating Python bindings. You need to know where the memory allocation for each object is and ensure that it is freed only on the same side of the language barrier.

For example, when you set x = 3. Memory for this Python side allocation, garbage collection is required. Fortunately, with Python objects, it’s hard to do anything else. Look at the reverse in C, where you allocate a block of memory directly:

int* iPtr = (int*)malloc(sizeof(int)); When you do this, you need to make sure that the pointer is freed in C. This might mean manually adding code to the Python binding to do this.

This completes your list of general topics. Let’s start setting up your system so you can write some code!

Set up your environment

In this tutorial, you’ll use the pre-existing C and C++ libraries from the Real Python GitHub repository to demonstrate tests for each tool. The goal is that you will be able to apply these ideas to any C library. To follow all the examples here, you need the following:

Knowledge of installed C++ libraries and command line call paths
Python development tools:
For Linux, this is the python3-dev or python3-devel package, depending on your distribution.
For Windows, there are several options.
Python 3.6 or higher
A virtual environment (recommended, but not required)
The invoke tools

The last one may be new to you, so let’s take a closer look at it.

Use the Invoke tool

Invoke is the tool you will use to build and test Python bindings throughout this tutorial. It has a similar purpose to make but uses Python instead of Makefiles. You need Invoke to install it in the virtual environment using the following PIP commands:

$python3-m PIP install invoke To run it, type invoke followed by the task to execute:

$ invoke build-cmult
==================================================
= Building C Library
* Complete

To see which tasks are available, use the following –list option:

$ invoke --list Available tasks: all Build and run all tests build-cffi Build the CFFI Python bindings build-cmult Build the shared library for the sample C code build-cppmult Build the shared library for the sample C++ code build-cython Build the cython extension module build-pybind11 Build the pybind11 wrapper library clean Remove any built objects test-cffi Run the script to test  CFFI test-ctypes Run the script to test ctypes test-cython Run the script to test Cython test-pybind11 Run the script to test PyBind11

Note that when you look at the tasks.py file that defines the task invoke, you will see that the name of the second task listed is build_cffi. However, the output from it displays its –list as build-cffi. The minus sign (-) cannot be used as part of a Python name, so the file uses an underscore (_) instead.

For each tool you will examine, a build- and a test- task will be defined. For example, to run the code CFFI, you can type invoke build-cffi test-cffi. One exception is ctypes, because there are no build-phase ctypes. In addition, for convenience, two special tasks have been added:

Invoke All runs the build and test tasks for all the tools.
Invoke Clean removes any generated files.

Now that you have an idea of how to run the code, let’s take a look at the C code you’ll be wrapping before looking at the tooling overview.

C or C++ source code

In each of the following sample sections, you’ll create Python bindings for the same function in C or C++. These sections are intended to give you a taste of what each method looks like, rather than an in-depth tutorial on the tool, so the functions you’ll encapsulate are small. The function for which you will create a Python binding takes anint and convertible as input parameters and returns afloat that is the product of two numbers:

// cmult.c
float cmult(int int_param, float float_param) {
    float return_value = int_param * float_param;
    printf("    In cmult : int: %d float %.1f returning  %.1f\n", int_param,
            float_param, return_value);
    return return_value;
}

C and C++ functions are almost identical, with slightly different names and strings between them. You can get a copy of all the code by clicking on the following link:

Now that you’ve cloned the repo and installed the tools, you can build and test them. So, let’s dig into each of the sections below!

ctypes

You’ll start with ctypes, which is the standard library tool for creating Python bindings. It provides a low-level toolset for loading shared libraries and marshalling data between Python and C.

How is it installed

The big advantage of ctypes is that it is part of the Python standard library. It was added in Python 2.5, so you probably already have it. You can import as if you were using the sys or time modules.

Call a function

All the code that loads the C library and calls the functions will be in the Python program. This is great because there are no extra steps in your process. You just run your program and everything will be taken care of. To create Python binding cTypes in, you need to perform the following steps:

Load your library.
Wrap some input parameters.
Tells ctypes the return type of your function.

You’ll look at each of them in turn.

Library to load

Ctypes gives you a variety of ways to load shared libraries, some of which are platform-specific. For your example, you use ctypes.cdll to create the object directly by passing in the full path to the desired shared library:

# ctypes_test.py
import ctypes
import pathlib

if __name__ == "__main__":
    # Load the shared library into ctypes
    libname = pathlib.Path().absolute() / "libcmult.so"
    c_lib = ctypes.CDLL(libname)

This works when shared libraries are in the same directory as Python scripts, but be careful when trying to load libraries from packages other than Python bindings. There is a lot of detail about loading libraries and finding paths in the platform-specific and situation-specific documentation of ctypes.

Note: There are many platform-specific issues that can occur during library loading. It is best to make incremental changes after the example works.

Now that you’ve loaded the library into Python, you can try calling it!

Call your function

Keep in mind that the function prototype for your C function looks like this:

// cmult.h
float cmult(int int_param, float float_param);

You need to pass in an integer and a floating point number, and you can expect a floating point number to return. Integers and floating-point numbers are natively supported in both Python and C, so you want this to apply to reasonable values.

When you load the library into the Python binding, this function will be the property c_lib, which is the object you created earlier on the CDLL. Try calling it something like this:

X, y = 6, 2.3 Answer = c_lib.cmult(x, y)

Oh dear! It doesn’t work. This line is commented out in the sample repo because it failed. If you try to run with this call, Python will report an error:

$ invoke test-ctypes
Traceback (most recent call last):
  File "ctypes_test.py", line 16, in <module>
    answer = c_lib.cmult(x, y)
ctypes.ArgumentError: argument 2: <class 'TypeError'>: Don't know how to convert parameter 2

It looks like you need to specify any arguments to ctypes that are not integers. Ctypes You don’t know anything about this function unless you explicitly tell it. Any parameters not otherwise marked are assumed to be integers. Ctypes does not know how to convert the values stored in 2.3 to y integers, so it fails.

To solve this problem, you need c_float to create one from the number. You can do this on the line where the function is called:

# ctypes_test.py
answer = c_lib.cmult(x, ctypes.c_float(y))
print(f"    In Python: int: {x} float {y:.1f} return val {answer:.1f}")

Now, when you run this code, it returns the product of the two numbers you passed in:

$invoke test-ctypes In cmult: int: 6 float 2.returning 13.8 In Python: int: 6 float 2.3 return val 48.0

Wait a minute… 6 times 2.3 is not 48.0!

It turns out that, just like the input parameter, ctypes assumes that your function returns an int. In fact, your function returns a float, which has been incorrectly marshalled. Just like the input parameter, you need to tell cTypes to use a different type. Here the syntax is slightly different:

# ctypes_test.py
c_lib.cmult.restype = ctypes.c_float
answer = c_lib.cmult(x, ctypes.c_float(y))
print(f"    In Python: int: {x} float {y:.1f} return val {answer:.1f}")

That should be enough. Let’s run the entire test-ctypes target and see what you have. Remember, the first part of the output is before restype fixed the function as a floating point number:

$ invoke test-ctypes ================================================== = Building C Library * Complete ================================================== = Testing ctypes Module In cmult : int: 6 float 2.3 returning 48.0 In cmult: int: 6 float 2.3 returning 48.0 In cmult: int: 6 float 2.3 returning 48.0 In cmult: int: 6 float 2.3 returning 13.8 In Python: int: 6 float 2.3 returning val 13.8

So much the better! Your fixed version is consistent with the C function, although the first uncorrected version returns the wrong value. Both C and Python get the same result! Now that it works, see why you might or might not want to use ctypes.

Strengths and weaknesses

The biggest advantage of ctypes over the other tools you’ll examine here is that it’s built into the standard library. It does not require additional steps, because all the work is done as part of a Python program.

In addition, the concepts used are low-level, which makes an exercise like the one you just did manageable. However, due to the lack of automation, more complex tasks become cumbersome. In the next section, you’ll see a tool that adds some automation to the process.

CFFI

CFFI is Python’s C foreign function interface. Generating Python bindings requires a more automated approach. CFFI has several ways to build and use Python bindings. There are two different options to choose from, giving you four possible modes:

ABI vs API: The API schema uses a C compiler to generate complete Python modules, while the ABI schema loads shared libraries and interacts directly with them. Getting the correct structure and parameters without running the compiler can be error-prone. The document strongly recommends using the API pattern.
Inline vs. Outreach: The difference between these two models is the tradeoff between speed and convenience:
- Inline mode compiles the Python binding every time the script is run. This is convenient because you don’t need additional build steps. However, it does slow down your program.
- The out-of-line pattern requires an additional step to generate Python bindings once and then use them each time the program runs. This is much faster, but it may not matter to your application.

For this example, you’ll use the API Outreach pattern, which produces the fastest code and generally looks similar to the other Python bindings you’ll create later in this tutorial.

How is it installed

Since CFFI is not part of the standard library, you need to install it on your machine. It is recommended that you create a virtual environment for this purpose. Fortunately, CFFI has PIP installed:

$python3-m PIP install cffi This will install the package into your virtual environment. You should be aware of this if you have already installed requirements.txt. You can see requirements.txt by visiting repo in the following link:

Get sample code: Click here to get the sample code that you will use to learn about Python bindings in this tutorial.

Now that you have CFFI installed, it’s time to give it a try!

Call a function

Instead of ctypes, CFFI you’re creating a full Python module. You will be able to import to use this module just like any other module in the standard library. You need to do some extra work to build the Python module. To use the Cffipython binding, you need to perform the following steps:

Write some Python code that describes the binding.
Run this code to generate a loadable module.
Modify the calling code to import and use the newly created module.

This may seem like a lot of work, but you’ll go through each of these steps and see how it works.

Write a binding

CFFI provides a way to read C header files to do most of the work when generating Python bindings. The code to do this is placed in a separate Python file. For this example, you’ll put the code directly into Invoke, the build tool, which uses a Python file as input. To use CFFI, you first create a CFFI.ffi object, which provides the three methods you need:

# tasks.py
import cffi
...
""" Build the CFFI Python bindings """
print_banner("Building CFFI Module")
ffi = cffi.FFI()

Once you have FFI, you will use.cdef() to automatically process the contents of the header file. This will create a wrapper function for you to marshal data from Python:

# tasks.py
this_dir = pathlib.Path().absolute()
h_file_name = this_dir / "cmult.h"
with open(h_file_name) as h_file:
    ffi.cdef(h_file.read())

Reading and processing header files is the first step. After that, you need to use.set_source() to describe the source file that CFFI will generate:

# tasks.py
ffi.set_source(
    "cffi_example",
    # Since you're calling a fully-built library directly, no custom source
    # is necessary. You need to include the .h files, though, because behind
    # the scenes cffi generates a .c file that contains a Python-friendly
    # wrapper around each of the functions.
    '#include "cmult.h"',
    # The important thing is to include the pre-built lib in the list of
    # libraries you're linking against:
    libraries=["cmult"],
    library_dirs=[this_dir.as_posix()],
    extra_link_args=["-Wl,-rpath,."],
)

Here is a breakdown of the parameters you passed in:

“Cffi_example” is the basic name of the source file that will be created on your file system. CFFI will generate a.c file, compile it into an.o file, and link it to a.

.so or.

.dll file.
‘#include “cmult.h”‘ is the custom C source, which will be included in the generated source before compilation. Here, you only need to include the file for which.h will generate the bindings, but this can be used for some interesting customizations.
Libraries =[“cmult”] tells the linker the name of your pre-existing C libraries. This is a list, so you can specify as many libraries as you need.
Library_dirs =[this_dir.as_posix(),] is a list of directories that tells the linker where to look for the above list of libraries.
The extra_link_args=[‘ -wl,-rpath,.’] is a set of options for generating shared objects that will be used in the current path (.) Find additional libraries it needs to load in.

Building Python Bindings

Calling.set_source() does not build a Python binding. It only sets the metadata to describe what will be generated. To build Python bindings, you need to call.compile() :

# tasks.py
ffi.compile()

This is done by generating.c files,.o files, and shared libraries. In Invoke you just walk through the task to build a Python binding from the command line:

$ invoke build-cffi
==================================================
= Building C Library
* Complete
==================================================
= Building CFFI Module
* Complete

You have your CFFIPython binding, so it’s time to run this code!

Call your function

After all the work you’ve done to configure and run the CFFI compiler, using the generated Python bindings will look just like using any other Python module:

# cffi_test.py import cffi_example if __name__ == "__main__": Print (f" In Python: int:) print(f" In Python: int: {x} float {y:.1f} return val {answer:.1f}")

You import the new module and then cmult() can be called directly. To test it, use the following test-cffi task:

$ invoke test-cffi ================================================== = Testing CFFI Module In cmult : int: 6 float 2.3 returning 13.8 In Python: int: 6 float 2.3 returning val 13.8

This will run your cffi_test.py program, which will test your use of CFFI. This concludes the section on writing and using Cffipython bindings.

Strengths and weaknesses

This seems to require less work for CTypes than the example you just saw for CFFI. While this is true for this use case, CFFI can scale better to larger projects than CTypes due to automation of most of the functionality wrapped.

CFFI also produces a completely different user experience. Ctypes allows you to load pre-existing C libraries directly into your Python programs. CFFI, on the other hand, creates a new Python module that can be loaded like any other Python module.

More importantly, using the external API approach used above, the time loss for creating a Python binding is done once when you build it, and does not occur every time you run the code. This may not be a big deal for small programs, but it can be better scaled up to larger projects using CFFI.

Like ctypes, usingCFFI only allows you to interact directly with C libraries. The C++ library requires a lot of work to use. In the next section, you’ll see a Python binding tool that focuses on C++.

PyBind11

PyBind11 takes a completely different approach to creating Python bindings. In addition to shifting the focus from C to C++, it uses C++ to specify and build modules that can take advantage of metaprogramming tools in C++. Like CFFI, the generated Python binding PyBind11 is a complete Python module that can be imported and used directly.

PyBind11 is based on the Boost::Python library and has a similar interface. However, it restricts its use to C++11 and later versions, which makes it easier and faster to process than Boost, which supports everything.

The “First Step” section of the documentation on how to install PyBind11 guides you through how to download and build PyBind11. While this may not seem strictly required, completing these steps will ensure that you have the correct C++ and Python tools set up.

Note: Most of the examples PyBind11 use CMake, which is a good tool for building C and C++ projects. For this demonstration, however, you will continue to use the Invoke tool, which follows the instructions in the manual build section of the documentation.

You need to install this tool into your virtual environment:

$python3-m PIP install pybind11 Pybind11 is a full-header library similar to most of Boost. This allows PIP to install the actual C++ source code of the library directly into your virtual environment.

Call a function

Before you delve further, note that you are using a different C++ source file, cppmult.cpp, rather than the C file you used for the previous example. The functions of the two languages are basically the same.

Write a binding

Like CFFI, you need to create some code that tells the tool how to build your Python bindings. Unlike CFFI, this code will use C++ instead of Python. Fortunately, very little code is required:

// pybind11_wrapper.cpp
#include <pybind11/pybind11.h>
#include <cppmult.hpp>

PYBIND11_MODULE(pybind11_example, m) {
    m.doc() = "pybind11 example plugin"; // Optional module docstring
    m.def("cpp_function", &cppmult, "A function that multiplies two numbers");
}

Let’s look at it one at a time, because PyBind11 packages a lot of information into a few lines.

The first two lines contain the files for the pybind11.hc ++ library and the header file cppmult.hpp. After that, you have the PYBIND11_MODULE macro. This will extend to the C++ code block detailed in the PyBind11 source code:

This macro creates the entry point that will be called when the Python interpreter imports the extension module. The module name is given as the first argument and should not be enclosed in quotes. The second macro defines a type variable that py::module can use to initialize the module. (source)

What this means to you is that, in this case, you are creating a module named pybind11_example, and the rest of the code uses m as the name of the py::module object. On the next line, in the C++ function that you define, you create a docstring for the module. While this is optional, making your module more Pythonic is a good choice.

Finally, you have the m.def() number. This defines a function that is exported by your new Python binding, which means it will be visible in Python. In this example, you will pass three parameters:

Cpp_function is the exported name of the function you will use in Python. As this example shows, it does not need to match the name of the C++ function.
&cppmult Gets the address of the function to export.
“A function…” Is an optional docstring for a function.

Now that you have the code for the Python binding, let’s look at how to build it into a Python module.

Building Python Bindings

The tool used to build Python bindings, PyBind11, is the C++ compiler itself. You may need to change the default values for the compiler and operating system.

First, you must build the C++ library for which you want to create bindings. For such a small example, you can build the cppmult library directly into the Python binding library. However, for most real-world examples, you will have a pre-existing library to wrap, so you will build cppmult separately. A build is a standard call to the compiler to build a shared library:

# tasks.py
invoke.run(
    "g++ -O3 -Wall -Werror -shared -std=c++11 -fPIC cppmult.cpp "
    "-o libcppmult.so "
)

Run this invoke build-cppmult to produce libcppmult.so:

$ invoke build-cppmult
==================================================
= Building C++ Library
* Complete

On the other hand, the construction of Python bindings requires some special details:

1# tasks.py 2invoke.run( 3 "g++ -O3 -Wall -Werror -shared -std=c++11 -fPIC " 4 "`python3 -m pybind11 --includes` " 5 "-I /usr/include/ python3.7-i. "6 "{0}" 7 "-o {1} 'python3.7-config --extension-suffix' "8" -l. -lcppmult -Wl,-rpath,.".format(cpp_name, extension_name) 9)

Let’s go over it line by line. Line 3 contains fairly standard C++ compiler flags, indicating several details, including that you want to catch all warnings and treat them as errors, that you need shared libraries, and that you are using C++11.

Line 4 is the first step of magic. It calls the PyBind11 module to include PyBind11. You can run this command directly from the console to see what it does:

$pybind11 python3 - m - includes - I/home/jima /. Virtualenvs/realpython/include/python3.7 m - I/home/jima /. Virtualenvs/realpython/include/site/python3.7

Your output should be similar but show a different path.

In line 5 of the compile call, you can see that you also added the path includes for Python Dev. While it is recommended that you do not link to the Python library itself, the source code requires some code called python.h to work its magic. Fortunately, the code it uses is fairly stable in the Python version.

Line 5 is also used for -i. Add the current directory to the include path list. This allows #include <cppmult.hpp> to parse lines in the wrapper code.

Line 6 specifies the name of the source file, which is pybind11_wrapper.cpp. Then, in line 7, you see more build magic happening. This line specifies the name of the output file. Python has some special ideas about module naming, including the Python version, machine architecture, and other details. Python also provides a tool to help with this problem. Python3.7-config:

$python3.7 - config - the extension - suffix. Retaining 37 m - x86_64 - Linux - gnu. So

If you are using a different version of Python, you may need to modify this command. If you use a different version of Python or on a different operating system, your results may vary.

The last line of the build command, line 8, points the linker to the library you built earlier with libcppmult. This RPath section tells the linker to add information to the shared library to help the operating system libcppmult look it up at run time. Finally, you’ll notice that the format of the string is cpp_name and extension_name. You’ll use this function again when Cython builds a Python binding module in the next section.

Run this command to build the binding:

$ invoke build-pybind11
==================================================
= Building C++ Library
* Complete
==================================================
= Building PyBind11 Module
* Complete

That’s it! You’re already using PyBind11. It’s time to test it out!

Call your function

Similar to the CFFI example above, once you’ve done the heavy lifting of creating Python bindings, calling your function looks just like normal Python code:

# pybind11_test.py import pybind11_example if __name__ == "__main__": Print (f" In Python: int: function(x, y)) print(f" In Python: int: {x} float {y:.1f} return val {answer:.1f}")

Since your pybind11_example is used as the name of the module in the PYBIND11_MODULE macro, this is the name you imported. Cpp_function is the method you use to call it from Python in the call m.def() that you tell PyBind11 to export the cppmult function to.

You can also test it invoke:

$ invoke test-pybind11 ================================================== = Testing PyBind11 Module In cppmul: int: 6 float 2.3 returning 13.8 In Python: int: 6 float 2.3 returning val 13.8

This is what PyBind11 looks like. Next, you’ll learn when and why PyBind11 is the right tool for this job.

Strengths and weaknesses

PyBind11 focuses on C++ rather than C, which makes it different from ctypes and CFFI. It has several features that make it very attractive to C++ libraries:

It supports classes.
It handles polymorphic subclassing.
It allows you to add dynamic properties to objects from Python and many other tools, which is difficult to do with the C-based tools you examined.

Having said that, you need to do a lot of setup and configuration to get PyBind11 up and running. Correct installation and build can be a little picky, but once done, it seems pretty reliable. In addition, PyBind11 requires that you use C++11 or higher at least. This is unlikely to be a big limitation for most projects, but it may be a consideration for you.

Finally, the extra code you need to write to create a Python binding is written in C++, not in Python. This may or may not be your problem, but it is different than the other tools you see here. In the next section, you’ll continue to discuss Cython, which takes a completely different approach to this problem.

Cython

The method Cython needs to create Python bindings using a Python-like language to define the bindings, and then the generated C or C ++ code can be compiled into modules. There are several ways to use Cython. One of the most common is to use setupFrom Distutils. For this example, you’ll stick with the Invoke tool, which allows you to use the exact command to run.

How is it installed

Cython is a Python module that you can install into your virtual environment from PyPI:

$python3-m PIP install cython Again, if you have installed the requires.txt file into the virtual environment, it already exists. You can get a copy of requirements.txt by clicking the following link:

Get sample code: Click here to get the sample code that you will use to learn about Python bindings in this tutorial.

This should get you ready to work with Cython!

Call a function

To use the build Python binding Cython, you will follow the same step as for CFFI and PyBind11. You’ll write the bindings, build them, and then run Python code to invoke them. Cython can support both C and C++. For this example, you will use cppmult the library you used in the example above PyBind11.

Write a binding

The most common form of a declaration module in Cython is to use a.pyx file:

1# cython_example.pyx
 2""" Example cython interface definition """
 3
 4cdef extern from "cppmult.hpp":
 5    float cppmult(int int_param, float float_param)
 6
 7def pymult( int_param, float_param ):
 8    return cppmult( int_param, float_param )

There are two parts here:

Lines 3 and 4 tell Cython that you are using cppmult() from cppmult.hpp.
Lines 6 and 7 create a wrapper function, pymult(), to call cppmult().

The language used here is a special combination of C, C++, and Python. To Python developers, however, it looks fairly familiar, because the goal is to make the process easier.

Withcdef extern… Tell Cython that the following function declaration can also be found in the cppmult.hpp file. This is useful for ensuring that Python bindings are built from the same declarations as the C++ code. The second part looks like a normal Python function — because it is! This section creates a Python function cppmult that has access to C++ functions.

Now that you’ve defined Python bindings, it’s time to build them!

Building Python Bindings

The build process Cython is similar to the build process you used, PyBind11. You first run Cython on a.pyx file to generate a.cpp file. Once this is done, you can compile PyBind11 using the same functions used for:

1# tasks.py 2def compile_python_module(cpp_name, extension_name): 3 invoke.run( 4 "g++ -O3 -Wall -Werror -shared -std=c++11 -fPIC " 5 "`python3 -m pybind11 --includes` " 6 "-I /usr/include/ python3.7-i. "7 "{0}" 8 "-o {1} 'python3.7-config --extension-suffix' "9" -l. -lcppmult -Wl,-rpath,.".format(cpp_name, extension_name) 10 ) 11 12def build_cython(c): 13 """ Build the cython extension module """ 14 print_banner("Building Cython Module") 15 # Run cython on the pyx file to create a .cpp file 16 invoke.run("cython --cplus -3 cython_example.pyx -o cython_wrapper.cpp") 17 18 # Compile and link the cython wrapper library 19 compile_python_module("cython_wrapper.cpp", "cython_example") 20 print("* Complete")

You start by running Cython on your.pyx file. You can use several options on this command:

— Cplus tells the compiler to generate C++ files instead of C files.
-3 Switch Cython to generate Python 3 syntax instead of Python 2.
-o cython_wrapper.cpp Specifies the name of the file to be generated.

After the C++ file is generated, you can use the C++ compiler to generate Python bindings, just as you did for PyBind11. Note that the include call to generate additional paths using the PyBind11 tool is still in the function. There won’t be any harm here, as your source doesn’t need these.

Running the task Invoke in generates the following output:

$ invoke build-cython
==================================================
= Building C++ Library
* Complete
==================================================
= Building Cython Module
* Complete

You can see that it builds the cppmult library and then builds the Cython module to wrap it. Now you have the CythonPython binding. (Trying to say quick…) It’s time to test it out!

Call your function

The Python code that calls the new Python binding is very similar to the code used to test other modules:

1# cython_test.py 2import cython_example 3 4# Sample data for your call 5x, y = 6, 6 answer = cython_example.pymult(x, y) 8print(f" In Python: int: {x} float {y:.1f} return val {answer:.1f}")

Line 2 imports the new Python binding module and pymult() is called on line 7. Remember that the.pyx file provides a Python wrapper cppmult() and renames it to pymult. Running your tests using Invoke yields the following results:

$ invoke test-cython ================================================== = Testing Cython Module In cppmul: int: 6 float 2.3 returning 13.8 In Python: int: 6 float 2.3 returning val 13.8

You get the same result as before!

Strengths and weaknesses

Cython is a relatively complex tool that gives you a deeper level of control when creating Python bindings for C or C++. While you won’t go into depth here, it provides a Python-style way to write code that manually controls the GIL, which can significantly speed up the processing of certain types of problems.

However, this Python-style language is not quite Python, so there is a slight learning curve when it comes to quickly determining which parts of C and Python fit where.

conclusion

A: congratulations! You now have an overview of several different options for creating Python bindings. You’ve learned about marshalling data and the issues to consider when creating bindings. You’ve seen how to call C or C++ functions from Python using the following tools:

ctypes
CFFI
PyBind11
Cython

You now know that while cTypes allows you to load DLLs or shared libraries directly, the other three tools require an extra step but still create full Python modules. As a bonus, you also used Invoke’s tool for running command-line tasks from Python.

Click on the attention, the first time to understand Huawei cloud fresh technology ~

Python bindings: Call C or C++ from Python

Summary of the Python Bindings

Marshal data types

Understand variable and immutable values

Memory management

Set up your environment

Use the Invoke tool

C or C++ source code

ctypes

How is it installed

Call a function

Library to load

Call your function

Strengths and weaknesses

CFFI

How is it installed

Call a function

Write a binding

Building Python Bindings

Call your function

Strengths and weaknesses

PyBind11

Call a function

Write a binding

Building Python Bindings

Call your function

Strengths and weaknesses

Cython

How is it installed

Call a function

Write a binding

Building Python Bindings

Call your function

Strengths and weaknesses

Other solutions

PyBindGen

Boost.Python

SIP

Cppyy

Shiboken

SWIG

conclusion

Related Posts

Web.py source code analysis: template (3)

Use pre-commit to uniformly manage hooks in your Git project

DRF usage of basename and source parsing