Run Rust in Android
Translator: iamazy
The original
For one of my current clients, we decided to make Rust our primary programming language. There are a number of reasons for this decision: aside from the technical advantages, there’s also the indisputable fact that Rust is still a relatively new language, fancy and trendy — and when you’re a startup, using technology that’s more than a decade old can get you bogged down. I mean – how do you innovate without using innovative technology? The quickest way to succeed is to hype it up.
“Users owning their own data” should be a selling point of the product, not a service that is completely accessible through a browser, but something that can be distributed to users and run on their devices. We’ve run some examples of Headless internally, and with a little more work, we can make redistributable packages for Windows and Linux. But we knew that if the package only ran on the desktop operating system, it would seriously hinder the adoption of the application – we needed a mobile version of the application if we wanted it to stand out. This means we need to know how to get our apps to run on Android or iOS. Because I already had some experience with cross-compilation and automated builds, I took the initiative to research this topic.
Access tools
To start with the basics, I need to get the Rust cross-compiler. Fortunately, Rust makes this fairly easy, as you only need to invoke the following command:
$ rustup target add armv7-linux-androideabi # For 32-bit ARM.
$ rustup target add aarch64-linux-android # For 64-bit ARM.
# x86_64 is mainly useful for running your app in the emulator.
# Speaking of hardware, there are some commercial x86-based tablets,
# and there's also hobbyists running Android-x86 on their laptops.
$ rustup target add x86_64-linux-android
Copy the code
(Note: Only all examples of the AARCH64 architecture will be shown later.)
I also need Android build tools. After some research, I went to the Android Studio download page and grabbed the archived command line tools. Although the SDK package is 80+ MiB in size, it is still only a minimal subset of the tools required, so I followed the advice of the Internet and used sdkManager to install the additional parts.
$ cd ~/android/sdk/cmdline-tools/bin/ $ ./sdkmanager --sdk_root="${HOME}/android/sdk" --install 'build-tools; /sdkmanager --sdk_root="${HOME}/android/ SDK "--install 'cmdline-tools; latest' $ ./sdkmanager --sdk_root="${HOME}/android/sdk" --install 'platform-tools' $ ./sdkmanager --sdk_root="${HOME}/android/sdk" --install 'platforms; android-29'Copy the code
Although Android supports running native code, most apps are written in Java or Kotlin, and the SDK reflects this. To be able to use Native code, I also needed a tool — the Native Development Kit. The NDK download page offers several versions to choose from – after much deliberation I decided to use the LTS version: R21E.
Simple enough! Or thinking too much?
With the development tools in hand, I decided to try compiling the project directly.
$ cargo build --target=aarch64-linux-android
Copy the code
As expected, the build failed and error messages filled the screen. After filtering, a link error appears:
error: linking with `cc` failed: exit code: 1
/usr/bin/ld: startup.48656c6c6f20546865726521.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: startup.48656c6c6f20546865726521.o: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status
Copy the code
I think this is simple enough – Cargo is trying to use the system’s linker instead of the Android NDK’s. I can use CC and LD environment variables to make Cargo point to the correct linker.
$ export ANDROID_NDK_ROOT="${HOME}/android/ndk"
$ export TOOLCHAIN="${ANDROID_NDK_ROOT}/toolchains/llvm/prebuilt/linux-x86_64"
$ export CC="${TOOLCHAIN}/bin/aarch64-linux-android29-clang"
$ export LD="${TOOLCHAIN}/bin/aarch64-linux-android-ld"
$ cargo build --target=aarch64-linux-android
Copy the code
To my disappointment, it didn’t work. I didn’t want to spend a day wrestling with Cargo, so I decided to see if anyone else had a solution — and pretty soon, I found what seemed like the perfect tool.
cargo-apk
Cargo – APK is a simple tool to build cargo projects into. Apk. All you need to do is install the tool, add some configuration to the Cargo. Toml file, and you’re good to go.
# cargo-apk compiles your code to an .so file,
# which is then loaded by the Android runtime
[lib]
path = "src/main.rs"
crate-type = ["cdylib"]
# Android-specic configuration follows.
[package.metadata.android]
# Name of your APK as shown in the app drawer and in the app switcher
apk_label = "Hip Startup"
# The target Android API level.
target_sdk_version = 29
min_sdk_version = 26
# See: https://developer.android.com/guide/topics/manifest/activity-element#screen
orientation = "portrait"
Copy the code
With the configuration added above, I tried to build the project using cargo- APk.
$ cargo install cargo-apk
$ export ANDROID_SDK_ROOT="${HOME}/android/sdk"
$ export ANDROID_NDK_ROOT="${HOME}/android/ndk"
$ cargo apk build --target aarch64-linux-android
Copy the code
Amazingly, it worked! (Wait) Uh, well, I’ve got a link error again. But this time, instead of a mysterious error about relocation and file format, it’s a missing link library:
error: linking with `aarch64-linux-android29-clang` failed: exit code: 1
aarch64-linux-android/bin/ld: cannot find -lsqlite3
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Copy the code
Dependence, dependence, dependence
Our project uses SQLite, a C library. Although the Rust community is notorious for touting “rewriting with Rust” on every possible occasion, in fact some Crate used with popular libraries does not need to be re-implemented because it requires a lot of work. Instead, they simply provide a way to call the library in Rust code, either by re-exporting it as a C function, or by providing a friendlier API and slightly abstracting FFI calls. The RusQLite we use is no different, which means we need to build SQLite as well.
SQLite is built using GNU Autotool. After learning a little about environment variables and the options for configuration, I perused the NDK’s documentation – I found a page of documentation that uses NDK in various build systems, including Autotools. Although Google provides the LTS version of the NDK, along with documentation for the latest version, things have changed between the R21 LTS and the latest R22, making things a little trickier. Fortunately, the Wayback machine had a historical version of the page that allowed me to find the proper NDK R21 instructions.
$ ANDROID_API=29$ TOOLCHAIN="${ANDROID_NDK_ROOT}/toolchains/llvm/prebuilt/linux-x86_64"i$ export CC="${TOOLCHAIN}/bin/aarch64-linux-android${ANDROID_API}-clang"$ export CXX="${TOOLCHAIN}/bin/aarch64-linux-android${ANDROID_API}-clang++"$ export AR="${TOOLCHAIN}/bin/aarch64-linux-android-ar"$ export AS="${TOOLCHAIN}/bin/aarch64-linux-android-as"$ export LD="${TOOLCHAIN}/bin/aarch64-linux-android-ld"$ export RANLIB="${TOOLCHAIN}/bin/aarch64-linux-android-ranlib"$ export STRIP="${TOOLCHAIN}/bin/aarch64-linux-android-strip"$ ./configure --host=aarch64-linux-android --with-pic$ make -j $(nproc)
Copy the code
Pick me up, Scotty
Using the above method, SQLite was successfully built and libsqlite3.so was generated. Now you just need to know how to get Cargo to use it. While browsing through the Cargo Book, I came across a section on environment variables that mentioned RUSTFLAGS. Just as Make or CMake treats CFLAGS and CXXFLAGS, the contents of RUSTFLAGS are passed by Cargo to the RUSTC compiler, allowing it to influence the compiler’s behavior.
While this approach was simple, it wasn’t very elegant for me, so I delved further into other options. Continuing through the Cargo Book, I came across the section that describes the configuration of the project, and I was sure that there was a way to specify RUSTFLAGS. However, no matter how hard I tried, I kept getting prompts from Cargo telling me about the unused MANIFEST key.
warning: unused manifest key: target.aarch64-linux-android.rustflags
Copy the code
Browsing through more chapters in the Cargo Book, I came across a section on build scripts. They no doubt is a powerful tool, but I’ve spent a lot of time learning Cargo configuration, don’t want to spend more time reading about how to write the contents of the build script, so finally I chose environment variable solution, and may be trying to use the build scripts (impossible).
I typed the command into the terminal and anxiously watched its execution.
$ RUSTFLAGS="-L $(pwd)/sqlite-autoconf-3340000/.libs/" cargo apk build --target aarch64-linux-android
Copy the code
Again, it’s… It worked to some extent. Although the linker no longer interprets errors as missing link libraries, cargo APK cannot find the linker and add it to the final APK file.
'lib/arm64-v8a/libstartup.so'... Shared library "libsqlite3.so" not found.Verifying alignment of target/debug/apk/statup.apk (4)... 49 AndroidManifest.xml (OK - compressed) 997 lib/arm64-v8a/libstartup.so (OK - compressed)Verification succesfulCopy the code
When I hadn’t compiled libsqlite3.so, I went back to the previous step and perused the error message generated by the linker. The linker combines a number of target files in the target/ aARCH64-linux-Android /debug/deps directory. What happens if I put my dot so file here?
$ cp sqlite-autoconf-3340000/.libs/sqlite3.so target/aarch64-linux-android/debug/deps$ cargo apk build --target aarch64-linux-android
Copy the code
To my surprise, it worked!
'lib/arm64-v8a/libstartup.so'... 'lib/arm64-v8a/libsqlite3.so'... Verifying alignment of target/debug/apk/startup.apk (4)... 49 AndroidManifest.xml (OK - compressed) 997 lib/arm64-v8a/libstatup.so (OK - compressed)15881608 lib/arm64-v8a/libsqlite3.so (OK - compressed)Verification succesfulCopy the code
I now have a.apk file that I can install on my Android phone. What a great success!
The application and the Activity
After compiling Rust code into.apk, all we have left to do is figure out how to merge Rust code with the Java code that wrote the UI. I naively typed “How to combine APK” into DuckDuckGo. After reading the top few results, it becomes clear that this is not possible, at least not without a deeper understanding of how Android apps work. That’s not to say, however, that there aren’t other approaches, because the article suggests another approach – combining activities into an application.
If, like me, you’ve never developed Android before, you might be wondering “what an Activity is” : When you design an app, it’s called an “interface” or “view.” For example, in the shopping app:
- The landing page is an Activity
- The product search page is an Activity
- The details page for the selected product is an Activity
- The shopping cart page is an Activity
- The checkout page is an Activity
Each page here may contain some interactive element, such as the ubiquitous hamburger menu. Theoretically, you could put the entire application in a single Activity if you wanted to, but it’s a bit harder to develop. Of course, there’s a lot more to say about Activity, but it’s not relevant for now.
Let’s continue with Rust. While the solution to my problem is to combine activities into one application, I’m not sure how the.apk files built with Rust tie into all of this. After taking a closer look at the cargo APk code, I realized that it was essentially wrapping my code into some glue code and creating NativeActivity for Android to run.
To combine activities into an application, I need to modify the application’s Androidmanifest.xml file to add the appropriate Activity nodes to the document. But how do I know the properties of the NativeActivity generated by cargo apk? Fortunately, when cargo- APk works, it generates a minimal version of the AndroidManifest.xml file and places it next to the generated.apk. The declaration of NativeActivity is as follows:
<activity android:name="android.app.NativeActivity" android:label="startup" android:screenOrientation="portrait" android:launchMode="standard" android:configChanges="orientation|keyboardHidden|screenSize"> <meta-data android:name="android.app.lib_name" android:value="startup" /> <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LAUNCHER" /> </intent-filter></activity>
Copy the code
All I have to do is copy and paste the code snippet above into the Manifest of the Java application.
Of course, this simply adds a statement to the application’s manifest that tells the application which activities to include. The Java application build process does not know the location of the libstartup.so file and automatically includes it. Luckily, ALL I had to do was copy the library files to the specified folder and Gradle automatically collected them.
$ mkdir -p android/app/src/main/jniLibs/arm64-v8a$ cp sqlite-autoconf-3340000/.libs/libsqlite3.so android/app/src/main/jniLibs/arm64-v8a/$ cp target/aarch64-linux-android/debug/libstatup.so android/app/src/main/jniLibs/arm64-v8a/$ cd android/ && ./gradlew && ./gradlew build
Copy the code
With that all done, I launched the build and it worked! I installed.apk on my idle Android device, but… There seems to be something wrong!
Once my app is installed, two shortcuts appear on the app’s startup screen. One starts the Java UI interface, while the other starts the NativeActivity containing Rust code. After reading more about the Activity and AndroidManifest, I learned that the part that causes this problem is the NativeActivity – that is, the category node declaration that should display it in the launcher. Once I remove it, everything returns to normal and NativeActivity is no longer displayed in the launcher.
However, one problem remains: how do I get a Java Activity to require Rust’s Activity to work for it?
Malicious Intent
Activities in Android can start with each other without a problem – if that’s not possible, you can’t really pass user information between the two. The standard way to call another Activity is through the startActivity() method, which takes one parameter: an instance of the Intent class.
Although the name of the Intent class is self-explanatory, its use can be a little unintuitive at first. In its most basic form, it just contains a reference to the instance calling the Activity and a class handle to the Activity we are calling. Specifically, an Intent calls a Context. Activity is just one type of Context.
However, intents can also be used to communicate why an Activity calls another Activity (such as an action). They can be used to distinguish between, for example, “show something” and “edit something.” Or the data URI to operate on and its MIME type. In addition to get/set methods, intEnts can hold almost any amount of “extra” data, which is typically stored as key-value pairs.
Intents provide a standardized way to pass information between activities. The caller provides the called with all the information it needs to process its request, and it can receive as a return value another Intent containing all the requested information. There’s nothing wrong with writing code in Java, but what happens when you put Rust code into NativeActivity?
If you look at the inheritance tree, you can see that NativeActivity inherits the Activity – which means it has access to all of the Activity’s non-private methods. I can call getIntent() and get the data from the caller. In addition, since this is a Java method and I’m running in native code, I need to use JNI (Java Native Interface) to perform the function call. Unfortunately, NativeActivity doesn’t have any other mechanism for passing information or using intents. This was frustrating because it meant I had to work with JNI.
JNI trip
I’ve spent so much time at this point with no visible results, it’s frustrating. On the other hand, I realized that using JNI opened up some new possibilities – instead of using activities and intents, I could paste code into functions and communicate by calling parameters and returning values. With this new idea in mind, I started my research on JNI.
Because in Java, everything is an object, and code cannot exist outside of a class – native code must also be part of the class. Since I don’t need persistence, I just use static methods.
package com.startup.hip; public class RustCode { public static native void doStuff(); }Copy the code
Above is a minimal example of a Java class with a static method labeled native. With that, I need to implement the functionality. But how do I use function signatures correctly?
Fortunately, Java has the capability to generate C header files for JNI. Before Java SE9, it was a standalone tool – Javah; Later, it was incorporated into the main Javac compiler executable as the -h option. This option requires a directory argument to place the generated. H files. The usage is very simple.
$ javac -h ./ RustCode.java
Copy the code
Calling the above command creates a com_startup_hip_rustcode.h file that contains the function definition.
#include <jni.h>JNIEXPORT void JNICALL Java_com_startup_hip_RustCode_doStuff(JNIEnv *, jclass);
Copy the code
With this knowledge, I can move on to creating the appropriate functions in Rust.
C + + flashback
When dealing with external code, Rust, much like C, mainly uses extern blocks. Also, like C++, Rust can use name mangling – which is not surprising given the language’s strong support for generics and macros. Fortunately, Rust provides an easy way to disable name mangling – using the [#no mangle] annotation.
use jni::{objects::JClass, JNIEnv}; #[no_mangle]pub extern "C" fn Java_com_startup_hip_RustCode_doStuff( _env: JNIEnv, _class: JClass,) {}
Copy the code
Having created the function declaration, I then need to write the corresponding implementation.
Receive parameters
In general, native functions need to accept some parameters. In this case, it is a string containing code that is then passed to the server.
package com.startup.hip; public class RustCode { public static native void doStuff(String code); }Copy the code
After modifying the Java code, I regenerated the C header file and edited the Rust code accordingly.
use jni::{objects::JClass, JNIEnv}; #[no_mangle]pub extern "C" fn Java_com_startup_hip_RustCode_doStuff( _env: JNIEnv, _class: JClass, code: JString,) {}
Copy the code
That’s easy. Now I need to extract the text from the Java string and pass it to the Rust code. This is more complicated than I expected. The problem is that the JVM internally stores strings using a modified version of UTF-8, and Rust strings must be valid UTF-8. Although Rust has a type for handling arbitrary strings, our code uses only the “classic” string type, and changing it all requires a lot of work.
Fortunately, the JNI library comes with a built-in mechanism for converting between standard UTF-8 and JVM-modified UTF-8 via a special JNIStr type. After perusing the documentation, I came up with the following code:
// Convert from JString -- a thinly wrapped JObject -- to a JavaStrlet code_JVM = env.get_string(code).unwrap(); // Convert from JString -- a thinly wrapped JObject -- to a JavaStrlet code_JVM = env.get_string(code). // Create a String from JavaStr, causing text conversionlet code_rust = String::from(code_jvm);Copy the code
Now I have a Rust string that I can pass to subsequent Rust code. Another great success!
The return value
The receiving argument is only half the story. I also need a return value, which incidentally is also a string – a string representing the return value of the server.
package com.startup.hip; public class RustCode { public static native String doStuff(String code); }Copy the code
Once again, I modified the Java code, regenerated the C header file, and edited the Rust code accordingly.
use jni::{objects::JClass, JNIEnv}; #[no_mangle]pub extern "C" fn Java_com_startup_hip_RustCode_doStuff<'a>( env: JNIEnv<'a>, _class: JClass, code: JString,) -> JString<'a>{ // function body here}
Copy the code
As you can see, return values in JNI are still treated as return values. All that remains is to create the JString that holds the result. Like get_string(), the JNIEnv structure also contains a new_string() function that does exactly what the name implies.
// Copy-pasted from earlier snippet let code_rust = String::from(env.get_string(code_jni).unwrap()); let result = match some_rust_function(code_rust) { Ok(value) => format! ("OK {}", value), Err(e) => format! ("ER {:? }", e),}; return env.new_string(result).unwrap();Copy the code
Just like that, my JNI wrapper is complete. Now I can call the Rust function in Java code, pass the value to the call and receive the return value.
Error handling by Rust
Although the code performed as expected, I didn’t like the number of.unwrap() calls. After all, error handling is an important part of Rust, and just because I’m interoperating with languages doesn’t mean I can ignore it. On the contrary, I believe that the interface between the two languages should be as simple as possible to prevent obscure errors from being discovered as a result of poor interoperability. Also, the Java return value must be checked to determine whether the call was successful, which makes the whole process somewhat unwieldy to use.
Instead of reinventing the wheel, I thought about how best to translate Rust’s Result method into Java side code. Fortunately, my Rust functions all return strings. As for errors, most errors are either unrecoverable or caused by bad input – which means I can forgo using exact error code and just rely on properly formatted error messages – which again refers to strings. So Result
can become Result
.
,>
Define a Java class
Although Java supports generics (albeit a bit of a cheat), I don’t want to delve into the details of using generics from JNI. I decided to create a Java class that roughly represents the Result
semantics.
public class Result { private boolean ok; private String value; public Result(boolean is_ok, String value) { this.ok = is_ok; this.value = value; } public boolean isOk() { return this.ok; } public boolean isError() { return ! this.ok; } public String getValue() { return this.ok ? this.value : null; } public String getError() { return this.ok ? null : this.value; }}Copy the code
Although this is done, it has some disadvantages compared to Rust – the most serious being that it returns NULL when accessing the wrong result variable. Since NULL is a non-problematic value for Java strings, calling getValue() can result in a NullPointerException popping up in inconsequential code without notice and passing it elsewhere. Although I could easily fix the problem by throwing an exception, I decided to do it the best way possible so that I never need to change this part of the code again.
Returns an object from JNI
The only thing left is to return an instance of the Result class from the Rust function. After some searching, I found a JNI function called NewObject(). This function takes four arguments:
- Handle to the JNI environment
- Handle to the class we want to create
- Constructor signature
- Arguments to the constructor
The Rust function takes a JNI environment handle as one of its parameters, so it is already processed. Constructor arguments can be passed as arrays, and I need to find two more function arguments.
To get a handle to this function, JNI provides the FindClass() function. It takes two parameters: the environment handle and the fully qualified name of the class – simply the “import name” of the class, but. Use/instead. For example, java.lang.String becomes Java /lang/String. In this case, com.startup.hip.Result becomes com/startup/hip/Result.
A constructor signature is a string that nicely describes how many arguments and what types are required for a constructor signature. At first glance, this was a bit confusing – but then I remembered that Java supports function overloading and includes constructors. Since a class can have multiple constructors, I have to let JNI know which constructor I want to use. After searching the Internet, I found that the easiest way to learn function signatures is to compile Java classes. Then use Java’s disassembly tool: Javap.
$ javac android/app/src/main/java/com/startup/hip/Result.java$ javap -s android/app/src/main/java/com/startup/hip/Result.classCompiled from "Result.java"public class com.startup.hip.Result { public com.startup.hip.Result(boolean, java.lang.String); descriptor: (ZLjava/lang/String;) V public boolean isOk(); descriptor: ()Z public boolean isError(); descriptor: ()Z public java.lang.String getValue(); descriptor: ()Ljava/lang/String; public java.lang.String getError(); descriptor: ()Ljava/lang/String; }Copy the code
Having executed the above command, I now know that the function signature I want to use is (ZLjava/lang/String;). V.
With all the steps in place, it’s time to create the array that holds the constructor parameters and call NewObject().
fn create_java_result<'e>( env: &JNIEnv<'e>, is_ok: bool, value: &str,) -> JObject<'e>{ let class = env .find_class("com/startup/hip/Result") .unwrap(); let args: [JValue<'e>; 2] = [ JValue::Bool(u8::from(is_ok)), JValue::Object(JObject::from(env.new_string(value).unwrap())), ]; env.new_object(class, "(ZLjava/lang/String;) V", &args) .unwrap()}Copy the code
Now I can return a custom Result Java class from a Native function.
Use a more generic solution
While the code above does this very well, it has one drawback: it explicitly takes booleans and strings, requiring the caller to handle Result himself and call the function with the appropriate parameters. Writing the logic that “errors should return as soon as possible” is tedious, but fortunately Rust provides a solution for this -? Operator. But our code calls functions from different libraries that use different error types — which means we can’t use Result
, And something like Result
must be performed – which is impossible because Rust does not allow features to be used as return types of functions.
The standard way to solve this problem is to use Box, but to make things easier, I decided to use the Anyhow library, which allows mixing and matching errors as I like. Anyway, I can write code like this:
fn rust_result_to_java_result<'e, T>( env: &JNIEnv<'e>, result: anyhow::Result<T>,) -> JObject<'e>where T: Display,{ let (is_ok, value) = match result { Ok(v) => (true, format! ("{}", v)), Err(e) => (false, format! (" {:? }", e)), }; create_java_result(env, is_ok, value)} fn actually_do_stuff<'a>( env: JNIEnv<'a>, code: JString,) -> anyhow::Result<String>{ let code = String::from(env.get_string(code)?) ; let intermediate_value = some_rust_function(code)? ; other_rust_function(intermediate_value)} #[no_mangle]pub extern "C" fn Java_com_startup_hip_RustCode_doStuff<'a>( env: JNIEnv<'a>, _class: JClass, code: JString,) -> JObject<'a>{ rust_result_to_java_result(actually_do_stuff(env, code))}Copy the code
Easier! Now I can return any desired result and convert it to an instance of a Java class for Java code to use.
encapsulation
Running Rust on Android was not an easy task, but I was pleased with the solution I eventually found. We write code using the plain old Rust and compile it into a shared library, which the JVM loads at run time. Although JNI may seem intimidating at first glance, using this standardized solution means that neither Java code nor Gradle build systems care that our native code is written in Rust. Cross-compiling with Cargo is still a bit tricky, as it turns out that Cargo apk sets up a lot of environment variables to make the whole process work. Our code also relies on external libraries – but all of that can be done with a bunch of shell scripts.
If you want to try it out for yourself, I’ve prepared a public Github repository that contains a minimal Android application that includes parts written in Rust and relies on external C libraries. The license for this project is Zlib. So feel free to grab the source code and use it for your own personal purposes.
reference
- Android NDK documentation: other build systems: Autoconf
- crates.io: cargo-apk
- cargo-apk: ndk-glue/src/lib.rs
- cargo-apk: nkd-build/src/cargo.rs
- Android developer documentation: app manifest:
- Android developer documentation: Activity
- Android developer documentation: NativeActivity
- Android developer documentation: Intent
- crates.io: jni
- Java SE 11: JNI specification
- Java SE 9: tools: javah
- The Rust Programming Language: Calling Rust Functions from Other Languages
- Java SE 11: tools: javap
- Thorn Technologies: Using JNI to call C functions from Android Java
- Code Ranch: How to create new objects with JNI
- [Stack Overflow: Java signature for method](