C/C++ to build the system, I use Xmake

What is XMake

XMake is a lua-based modern C/C++ build system.

Its syntax is simple and user-friendly, it’s easy to get started even if you don’t know Lua at all, and it’s completely dependency free, lightweight, and cross-platform.

It is also a self-fulfilling build system with a powerful package management system and a fast build engine.

Compared to Ninja/Scons/Make as Build Backend and CMake/Meson as Project Generator, XMake is both plus a package manager.

xmake = Build backend + Project Generator + Package Manager
Copy the code

Therefore, with an XMake installation package of less than 3M, you can start your C/C++ development journey without installing any other tools, not even make, or heavyweight runtime environments such as Python or Java.

You might say, well, the compiler has to be installed. This is not necessary, because XMake’s package management also supports automatic remote pull of the various compilation toolchains needed, such as LLVM, Mingw, Android NDK, or cross-compilation toolchains.

Why make XMake

When discussing XMake in the Reddit community, the following image is often used to make fun of it.

Although somewhat frustrated and somewhat numb by the joke, I want to make it clear that the original intention of XMake was not to split the C/C++ ecosystem, but to reuse the existing ecosystem as much as possible.

It also gives users the same experience as other C/C++ projects, such as Rust/Cargo, Nodejs/Npm, Dlang/Dub, without having to search for third packages and figure out how to port and compile.

So don’t jump to conclusions if you don’t already know XMake, try it out first, or take some time to read the details below.

Features and advantages of XMake

I’m often asked what’s so special about XMake, what advantages it has over existing build tools like CMake and Meson, and why I’m using XMake instead of CMake.

First, XMake has the following features and advantages:

• Simple and easy-to-learn configuration syntax, non-DSL
• Powerful package management, support semantic version, tool chain management
• Lightweight enough, free of dependencies
• Super fast build, as fast as Ninja
• Simple and convenient multi-platform, tool chain switch
• Complete plug-in system
• Flexible build rules

As for CMake, it has become a de facto standard, with complete ecology and powerful functions.

I never intended XMake to replace it, and it certainly won’t, but CMake has a number of drawbacks, including complicated syntax and inadequate package management support.

Therefore, XMake can be used as a complement for those who want a quick and easy start to C/C++ development, or want more easy-to-use package management, or want to write short test projects quickly.

XMake can help them improve their development efficiency by focusing more on the C/C++ project itself, rather than spending more time building tools and development environments.

Now, LET me elaborate on these key features of XMake.

Syntax is simple and easy to use

CMake designs a DSL language for project configuration, which increases the learning cost for users. Moreover, its syntax readability is not very intuitive, and it is easy to write overly complex configuration scripts, which also increases the maintenance cost.

XMake reuses the well-known Lua language as its base and provides a simpler and more straightforward configuration syntax.

Lua itself is a simple lightweight glue language, keywords and built-in types of just a few, read an article, you can get a basic start, and compared to DSL, you can get a lot of related materials and tutorials more easily from the Internet.

Basic grammar

However, some people will joke: that is not still have to learn Lua?

XMake uses a separate description field and script field, so that novice users can configure the description field in a simpler and more direct way 80% of the time, without treating it as a Lua script. For example:

target("test")
    set_kind("binary")
    add_files("src/*.c")
    add_files("test/*.c"."example/**.cpp")
Copy the code

If it looks like a function call in a scripting language because of the parentheses, then we can do the same (parentheses are a matter of personal preference, but I personally recommend using the above method).

target "test"
    set_kind "binary"
    add_files "src/*.c"
    add_files "test/*.c"
    add_files "example/**.cpp"
Copy the code

We only need to know the common configuration interface, Lua can be configured quickly even if not completely.

We can compare the configuration of CMake:

add_executable(test "")
file(GLOB SRC_FILES "src/*.c")
file(GLOB TEST_FILES "test/*.c")
file(GLOB_RECURSE EXAMPLE_FILES "example/*.cpp")
target_sources(test PRIVATE
    ${SRC_FILES}
    ${TEST_FILES}
    ${EXAMPLE_FILES}
)
Copy the code

It’s easy to see which is more intuitive to read.

Conditions for configuration

If you already know the basics of Lua, such as if then, then you can do some more conditional configuration.

target("test")
    set_kind("binary")
    add_files("src/main.c")
    if is_plat("macosx"."linux") then
        add_defines("TEST1"."TEST2")
    end
    if is_plat("windows") and is_mode("release") then
        add_cxflags("-Ox"."-fp:fast")
    end
Copy the code

CMake version configuration:

add_executable(test "")
if (APPLE OR LINUX)
    target_compile_definitions(test PRIVATE TEST1 TEST2)
endif(a)if (WIN32)
    target_compile_options(test PRIVATE $<$<CONFIG:Release>:-Ox -fp:fast>)
endif(a)target_sources(test PRIVATE
    src/main.c
)
Copy the code

Complex scripts

If you’ve advanced to the top of the XMake game, Lua syntax is well understood and you want more flexibility in customizing your configuration needs, and a few simple lines of configuration describing the field won’t suffice.

XMake also offers more complete Lua script customization, allowing you to write any complex script.

For example, all source files should be preprocessed before construction, and external gradle commands should be executed after construction for later packaging. We can even rewrite internal linking rules to achieve deep custom compilation. We can import the built-in Linker extension module through import interface to achieve complex and flexible linking process.

target("test")
    set_kind("binary")
    add_files("src/*.c")
    before_build_file(function (target, sourcefile)
        io.replace(sourcefile, "#define HAVE_XXX 1"."#define HAVE_XXX 0")
    end)
    on_link(function (target)
        import("core.tool.linker")
        linker.link("binary"."cc", target:objectfiles(), target:targetfile(), {target = target})
    end)
    after_build(function (target)
        if is_plat("android" then
            os.cd("android/app")
            os.exec("./gradlew app:assembleDebug")
        end
    end)
Copy the code

If CMake is changed, add_custom_command can also be implemented in the add_custom_command, but it seems that the add_custom_command can simply execute some batch commands, not do all kinds of complex logic judgment, module loading, custom configuration scripts, etc.

Of course, you can do the same with Cmake, but it’s not that simple.

If you are familiar with Cmake, you can also try to help with the following configuration:

add_executable(test "")
file(GLOB SRC_FILES "src/*.c")
add_custom_command(TARGET test PRE_BUILD
    -- TODO
    COMMAND echo hello
)
add_custom_command(TARGET test POST_BUILD
    COMMAND cd android/app
    COMMAND ./gradlew app:assembleDebug
)
-- How can we override link stage?
target_sources(test PRIVATE
    ${SRC_FILES}
)
Copy the code

Powerful package management

As we all know, the most important aspect of C/C++ related project development is the integration of various dependency packages, because there is no comprehensive package management system like Rust/Cargo.

Therefore, every time we want to use a third-party library, we need to search, study the porting and compiling of various platforms, and often encounter various compilation problems, which greatly delays the developer’s time and can not concentrate on the actual project development.

There are some third-party package managers like VCPKG/Conan/Conda, but some don’t support semantic versions, some support limited platforms, but anyway, This is a big step towards resolving the dependency management of C/C++ libraries.

However, with package managers, using them in C/C++ projects is still a bit of a hassle, as they need to be well integrated with the build tools.

CMake and Vcpkg

Let’s take a look at the integration support for CMake and Vcpkg:

Cmake_minimum_required (VERSION 3.0) project(test) Find_package (unofficial- SQlite3 CONFIG REQUIRED) add_executable(main) main.cpp) target_link_libraries(main PRIVATE unofficial::sqlite3::sqlite3)Copy the code

Disadvantages:

• Additional configuration is required-DCMAKE_TOOLCHAIN_FILE=<vcpkg_dir>/scripts/buildsystems/vcpkg.cmake"
• Automatic installation of dependency packages is not supported. You need to manually install dependency packagesvcpkg install xxxCommand to install
• The semantic version of VCPKG is not supported.

CMake and Conan

' 'cmake cmake_minimum_required(VERSION 2.8.12) project(Hello) add_definitions("-std=c++11") include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake) conan_basic_setup() add_executable(hello hello.cpp) target_link_libraries(hello gtest)Copy the code

conanfile.txt

[requires]
gtest/1.10.0

[generators]
cmake
Copy the code

Disadvantages:

• Again, extra calls are requiredconan install ..To install the package
• You also need to configure an additional conanfile.txt file to describe the package dependency rules

Meson and Vcpkg

I couldn’t find how to use the VCPKG package in Meson, just an Issue #3500 discussion.

Meson and Conan

Meson does not appear to have to support Conan, but Conan the official documentation on the solution, aligned to support, but it’s complicated, I didn’t watch is, everyone can study: docs. Conan. IO/en/latest/r…

XMake and Vcpkg

With all that said, the integration of other build tools and package management is a bit of a hassle to use, and the integration varies greatly from package manager to package manager, with users wanting to switch from Vcpkg to Conan package very quickly.

Next, let’s take a look at the packages integrated with Vcpkg in XMake:

add_requires("vcpkg::zlib", {alias = "zlib"})
target("test")
    set_kind("binary")
    add_files("src/*.c")
    add_packages("zlib")
Copy the code

We can directly integrate zlib packages that use VCPKG by requiring the corresponding package name on the add_requires configuration and the VCPKG :: package namespace.

Then, we only need to execute the xmake command to complete the compilation process, including the automatic installation of the zlib package, without the additional manual execution of VCPKG install zlib.

$ xmake
note: try installing these packages (pass -y to skip confirm)?
-> vcpkg::zlib
please input: y (y/n)

=> install vcpkg::zlib .. ok
[ 25%]: compiling.release src\main.cpp
[ 50%]: linking.release test
[100%]: build ok!
Copy the code

XMake and Conan

The next step is to integrate Conan’s package in exactly the same way, just implement a different package manager name.

add_requires("conan::zlib", {alias = "zlib"})
target("test")
    set_kind("binary")
    add_files("src/*.c")
    add_packages("zlib")
Copy the code

XMake also automatically installs the Zlib package in Conan, and then automatically integrates it.

XMake self-built package management

One of the biggest differences between XMake and other build systems like CMake, and one of the biggest advantages, is that it has a completely self-built package management system, so we can completely not rely on VCPKG/Conan, and we can quickly integrate dependencies like:

add_requires("zlib 1.2.x"."Tbox > = 1.6.0")
target("test")
    set_kind("binary")
    add_files("src/*.c")
    add_packages("zlib"."tbox")
Copy the code

Furthermore, it also supports full semantic version selection, multi-platform package integration, cross-compilation package integration, and even automatic pull use of compilation toolchain package.

Not only that, we can also customize the dependencies on the built packages, for example:

Rely on packages using modulated versions

We can use the Debug version library to debug breakpoints for dependent libraries.

add_requires("zlib", {debug = true})
Copy the code

Set up the MSVC runtime library

add_requires("zlib", {configs = {vs_runtime = "MD"}})
Copy the code

Using dynamic Libraries

Static libraries are integrated by default, but we can also switch to dynamic libraries.

add_requires("zlib", {configs = {shared = true}})
Copy the code

Semantic versioning support

XMake’s build-in package integration supports full version semantic specifications.

add_requires("zlib 1.2.x")
add_requires("Zlib > = 1.2.10")
add_requires("Zlib ~ 1.2.0")
Copy the code

Do not use system libraries

By default, if the version matches, XMake prioritises using libraries already installed by the user on the system, but we can also enforce a ban on using the system libraries and simply download the installation packages from our repository.

add_requires("zlib", {system = true})
Copy the code

Optional dependency packages

If the dependency integration fails, XMake will automatically report an error, interrupting the compilation and telling the user: Zlib not found, but we can also set it to optional integration, so that even if the library does not end up being installed, the project will not be affected and the dependency will just be skipped.

add_requires("zlib", {optional = true})
Copy the code

Package customization configuration

For example, the integration uses boost libraries with context/ Coroutine module configuration enabled.

add_requires("boost", {configs = {context = true.coroutine = true}})
Copy the code

Support for package management repositories

XMake in addition to supporting VCPKG/integration support conan and self-built warehouse package, also supports other package management of warehouse, for example: Conda/Homebrew/Apt/Pacman/Clib Dub, etc., and integrated approach.

Users can quickly switch to other repository packages without spending too much time figuring out how to integrate them.

Therefore, XMake does not destroy the C/C++ ecosystem, but greatly reuse the existing C/C++ ecosystem, and strive to improve the user experience of C/C++ dependent packages, improve the development efficiency, so that users can have more time to focus on the project itself.

• Xmake-repo (tbox >1.6.1)
• Official package manager Xrepo
• User – built warehouses
• Conan, Conan: : openssl / 1.1.1 g)
• Conda (Conda: : libpng 1.3.67)
• Vcpkg (vcpkg:ffmpeg)
• Homebrew/Linuxbrew (brew::pcre2/libpcre2-8)
• Pacman on archlinux/msys2 (pacman::libcurl)
• Apt on ubuntu/debian (apt::zlib1g-dev)
• Clib (Clib: : clibs/[email protected])
• Dub, Dub: : log 0.4.3)

Standalone package management command (Xrepo)

To facilitate package management in XMake’s own repository and third-party package management, we also provide a separate Xrepo CLI tool to facilitate the management of our dependent packages

We can use this tool to quickly and conveniently complete the following management operations:

• The installation package:xrepo install zlib
• Uninstall the package:xrepo remove zlib
• Get package information:xrepo info zlib
• Get package build link flags:xrepo fetch zlib
• Loading package virtual Shell environment:xrepo env shell(This is a powerful feature)

You can check out the Xrepo project home page for more information about how to use it.

Light weight without dependence

With Meson/Scons you need to install Python/PIP, and with Bazel you need to install Java and other runtime environments. XMake doesn’t need to install any additional dependent libraries and environments, and its own installation package is only 2-3m, which is very lightweight.

Although XMake is based on Lua, it is fully built in thanks to the lightweight nature of the Lua glue language, so installing XMake is equivalent to having a full Lua VM.

Some people would say that you need to build a toolchain anyway, but not entirely. On Windows, we’ve got a pre-compiled installation package that you can download and install directly, and the address is dependency Dependency

XMake also supports remote pull of the compiler chain, so even if you don’t have any compilers installed on your system, you don’t have to worry about messing with the compiler environment, just configure the required toolchain in XMake. Lua.

For example, we use mingw-W64 toolchain to compile C/C++ projects on Windows, and just do the following configuration.

add_requires("mingw-w64")
target("test")
    set_kind("binary")
    add_files("src/*.c")
    set_toolchains("mingw@mingw-w64")
Copy the code

After set_toolChains is configured to bind the mingW-W64 toolchain package, XMake will automatically detect the presence of mingW-64 in the current system. If mingW-64 is not installed, XMake will automatically download and install it, and complete the project compilation. All the user needs to do is execute the xmake command.

$ xmake
note: try installing these packages (pass -y to skip confirm)?
inXmake-repo: -> mingw-w64 8.1.0 [vs_Runtime :MT] Please input: y (y/n) => download https://jaist.dl.sourceforge.net/project/mingw-w64/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/8.1.0/ Threads - posix/seh/x86_64-8.1.0 - release - posix - seh - rt_v6 - rev0.7 z.. ok checkingformingw directory ... C: \ Users \ ruki \ AppData \ Local \. Xmake \ packages \ m \ mingw - w64\8.1.0 \ aad6257977e0449595004d7441358fc5 [25%] : compiling.release src\main.cpp [ 50%]: linking.release test.exe [100%]: build ok!Copy the code

In addition to mingW-W64, we can also configure remote pull using other toolchains, and even cross-compile toolchains, such as LLVM-mingw, LLVM, TinyCC, MuslCC, GNU-RM, ZIg, etc.

If you want to learn more about pull integration of remote toolchains, you can read the documentation: Automatic Pull Remote toolchains.

Extremely fast parallel compilation

Ninja is known for being very fast to build, so many people like to use CMake/Meson to generate builds. After that, Ninja can be used for extremely fast builds.

Although Ninja is fast, we need to generate build. Ninja from the meson.build and cmakelist. TXT files, which can take a few seconds or more.

Not only does XMake have nearly the same build speed as Ninja, it doesn’t need to regenerate additional build files, it’s built directly into the build system, and in any case, it can compile extremely fast with just one XMake command.

We have also done some comparative test data for your reference:

Multi-task, parallel compilation tests

Build system	Termux (8core/-j12)	Build system	MacOS (8core/-j12)
xmake	24.890 s	xmake	12.264 s
ninja	25.682 s	ninja	11.327 s
cmake(gen+make)	5.416 s + 28.473 s	cmake(gen+make)	1.203 s + 14.030 s
cmake(gen+ninja)	4.458 s + 24.842 s	cmake(gen+ninja)	0.988 s + 11.644 s

Single-task build tests

Build system	Termux (-j1)	Build system	MacOS (-j1)
xmake	1 m57. 707 s	xmake	39.937 s
ninja	1 m52. 845 s	ninja	38.995 s
cmake(gen+make)	5.416 s + 2 m10. 539 s	cmake(gen+make)	1.203 s + 41.737 s
cmake(gen+ninja)	4.458 s + 1 m54. 868 s	cmake(gen+ninja)	0.988 s + 38.022 s

Dumb multiplatform compilation

Another feature of XMake is that it is efficient and easy to compile on multiple platforms, whether you are building applications for Windows/Linux /macOS, iphoneos/android or cross-compiling.

The configuration of compilations is pretty much the same, so you don’t have to go through the hassle of figuring out how to compile for each platform.

Compile native Windows/Linux/MacOS programs

For current native program compilation, we only need to execute:

$ xmake
Copy the code

Contrast CMake

$ mkdir build
$ cd build
$ cmake --build ..
Copy the code

Compiling Android Applications

$ xmake f -p android --ndk=~/android-ndk-r21e
$ xmake
Copy the code

Contrast CMake

$ mkdir build
$ cd build
$ cmake -DCMAKE_TOOLCHAIN_FILE=~/android-ndk-r21e/build/cmake/android.toolchain.cmake ..
$ make
Copy the code

Compiling iOS apps

$ xmake f -p iphoneos
$ xmake
Copy the code

Contrast CMake

$ mkdir build
$ cd build
$ wget https://raw.githubusercontent.com/leetal/ios-cmake/master/ios.toolchain.cmake
$ cmake -DCMAKE_TOOLCHAIN_FILE=`pwd`/ios.toolchain.cmake ..
$ make
Copy the code

I didn’t find a convenient way to configure and compile ios programs. I just had to find a third-party ios toolchain somewhere to configure and compile.

cross-compilation

We usually only need to set up the root directory of the cross-compilation tool chain, and XMake automatically detects the structure of the tool chain and extracts the compiler from it to participate in the compilation, with no additional configuration required.

$ xmake f -p cross --sdk=~/aarch64-linux-musl-cross
$ xmake
Copy the code

Contrast CMake

We need to write an additional cross-toolchain-cmake configuration file.

set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)

set(TOOL_CHAIN_DIR ~/aarch64-linux-musl)
set(TOOL_CHAIN_INCLUDE ${TOOL_CHAIN_DIR}/aarch64-linux-musl/include)
set(TOOL_CHAIN_LIB ${TOOL_CHAIN_DIR}/aarch64-linux-musl/lib)

set(CMAKE_C_COMPILER "aarch64-linux-gcc")
set(CMAKE_CXX_COMPILER "aarch64-linux-g++")

set(CMAKE_FIND_ROOT_PATH ${TOOL_CHAIN_DIR}/aarch64-linux-musl)

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

include_directories(${TOOL_CHAIN_DIR}/aarch64-linux-musl/include)
set(CMAKE_INCLUDE_PATH ${TOOL_CHAIN_INCLUDE})
set(CMAKE_LIBRARY_PATH ${TOOL_CHAIN_LIB})
Copy the code

$ mkdir build
$ cdbuild $ cmake -DCMAKE_TOOLCHAIN_FILE=.. /cross-toolchain.cmake .. $ makeCopy the code

conclusion

If you are new to C/C++ development, you can use XMake to get started quickly with C/C++ compilation and construction.

If you want to develop and maintain a cross-platform C/C++ project, you can also consider using XMake to maintain your build, which will improve your development efficiency and allow you to focus more on the project and not have to worry about porting dependencies.

Welcome to XMake project:

• Github project address
• Project home page
• The XMake package manages the warehouse
• community
  • Telegram group
  • Discord chat room
  • QQ group: 343118190, 662147501
  • Wechat official account: Tboox-OS
• Course: Xmake makes it easy to build C/C++ projects
• Activity: Open Source Summer & Xmake