IO /flutter-ffi…

IO/asim.ihsan.io/

Published: June 21, 2020

The profile

Flutter is a Google UI toolkit for writing cross-platform mobile applications. By using the Dart: FFI library, which is currently in beta, it is now easier than ever to use the native code of Flutter. In this article, we’ll demonstrate why you should use native code and a practical example of using the libsodium encryption library.

By the end of this article, you will be able to.

  • Create a Flutter plugin that uses native code for iOS and Android mobile applications.
  • Use the libsodium native encryption library in the Flutter mobile app.
  • Run expensive native code in the background to avoid clogging the mobile app’s user interface.

The views expressed in this blog are my own and not necessarily those of my employer.

  1. introduce
  2. Existing technology, reference materials and other resources
  3. guide
    1. A prerequisite for
    2. Get libsodium
    3. Build libsodium for iOS
    4. Build libsodium for Android
    5. Create a Flutter plugin and bind it to libsodium using FFI.
      1. Flutter iOS plugin Settings
      2. Flutter Android plugin Settings
    6. Flutter Dart code — Triviality begins
    7. The Flutter Dart code is sealed before being dismantled.
  4. Reference code
  5. Future work and areas for improvement

introduce

Flutter is a software toolkit that allows you to write code and deploy applications to iOS and Android simultaneously. Dart, the programming language used by Flutter, is powerful and fast enough for most purposes. However, sometimes you want to use a pre-existing native library of code that has already been written and battle-tested in another programming language, such as because.

  1. You need to write code in a different, faster, more memory-saving language. For example, by writing a reusable library in Rust, you can take advantage of a more powerful optimized compiler and avoid the overhead of the garbage collector when needed.
  2. You want to reuse the same code in multiple areas, such as your Flutter mobile app, desktop app, and server. Although Google is beginning to introduce desktop and web support for Flutter, you can currently write in different languages in reusable libraries in order to share logic across multiple domains.
  3. You want to reuse existing code, especially for security-sensitive applications such as encrypting data. You don’t want to rewrite code like this because it might introduce errors, give away information, or cause your application to crash. Instead, by using a write-ahead library that has been approved by a security engineer, you’ll have more confidence that it will work.

In this article, we will use an example driven by these three reasons. Libsodium is an encryption library that allows you to encrypt, decrypt, and hash data for example. Libsodium is fast, approved by security engineers, and allows you to reuse the same code on your server, making it easier to decrypt encrypted data on your mobile device.

This article will start from scratch. We will create a Flutter plugin to compile libsodium for iOS and Android, and finally demonstrate how to use libsodium from Flutter and from the server. By reading this article, you’ll be able to use code from other native libraries, not just libsodium.

Existing technology, reference materials and other resources

This great article, also from scratch, shows you how to share libraries written by Rust with Android, iOS and Flutter. However, since this article does not use the new Dart: Fi feature, which is currently in beta, there is a lot of overhead as you need to write custom code in Swift and Kotlin to share the libraries with iOS and Android, respectively. I will write a follow-up article showing how easy it is to use a real-world Rust library example in Flutter.

Flutter_sodium is an existing Flutter plugin that binds to libsodium using the new DART: FI feature. I was motivated to write this post by reading the code for Flutter_sodium, but I made a few different implementation choices. In addition, flutter_sodium uses pre-built libsodium libraries, and this article will show you how to recompile libsodium from scratch.

For background on the Flutter plugin, see.

  • Development kits and plug-ins
  • Bind to native code using DART: FI

guide

A prerequisite for

I happen to work in $HOME/Programming, but change it wherever you like.

ROOT_DIR=$HOME/Programming
Copy the code

Download the Android the NDK, then ANDROID_NDK_HOME and NDK_HOME environment variables are set to $HOME/Library/Android/SDK/the NDK – bundle.

export ANDROID_NDK_HOME=$HOME/Library/Android/sdk/ndk-bundle
export NDK_HOME=$HOME/Library/Android/sdk/ndk-bundle
Copy the code

Make sure there aren’t any advanced issues with your Flutter installation, and make sure you get the new Dart FFI features on the Beta channel.

flutter channel beta
flutter upgrade
flutter doctor -v
Copy the code

To pretend to decrypt data sent by the Flutter mobile application as a server, we will use the Pynacl Python module. Use your Python system to install or install Python with.

  • Homebrew for Mac, or
  • All operating systems of Anaconda, or
  • Any other way you like.

Then install Pynacl and ipython (for a useful REPL shell).

pip install pynacl ipython
Copy the code

Get libsodium

As of 2020-06-14, V1.0.18 is the latest stable version of libsodium.

cd $ROOT_DIRWget https://download.libsodium.org/libsodium/releases/libsodium-1.0.18-stable.tar.gz tar XVF Libsodium 1.0.18 - stable. Tar. Gz rm -f libsodium - 1.0.18 - stable. Tar. GzCopy the code

Build libsodium for iOS

Libsodium provides easy-to-use build scripts for compiling iOS and Android libraries.

This will place the artifact in $ROOT_DIR/ libsoap-stable/libsoap-ios. We specified LIBSODIUM_FULL_BUILD so that we can expose all apis, not just the high-level ones.

cd $ROOT_DIR/libsodium-stable

# Clean up from previous builds
test -d libsodium-ios || rm -rf libsodium-ios
./configure && make distclean

LIBSODIUM_FULL_BUILD=true ./dist-build/ios.sh
Copy the code

If successful, you will see the path to a single binary file for all schemas.

libsodium has been installed into /Users/asimi/Programming/libsodium-stable/libsodium-ios

/Users/asimi/Programming/libsodium-stable/libsodium-ios/lib/libsodium.a: Mach-O universal binary with 5 architectures: [i386:current ar archive random library] [arm_v7:current ar archive random library] [arm_v7s] [x86_64] [arm64]
/Users/asimi/Programming/libsodium-stable/libsodium-ios/lib/libsodium.a (for architecture i386):	current ar archive random library
/Users/asimi/Programming/libsodium-stable/libsodium-ios/lib/libsodium.a (for architecture armv7):	current ar archive random library
/Users/asimi/Programming/libsodium-stable/libsodium-ios/lib/libsodium.a (for architecture armv7s):	current ar archive random library
/Users/asimi/Programming/libsodium-stable/libsodium-ios/lib/libsodium.a (for architecture x86_64):	current ar archive random library
/Users/asimi/Programming/libsodium-stable/libsodium-ios/lib/libsodium.a (for architecture arm64):	current ar archive random library
Copy the code

Build libsodium for Android

Again, we’ll build libraries for Android using the existing libsodium build scripts.

cd $ROOT_DIR/libsodium-stable

# Clean up from previous builds
./configure && make distclean

LIBSODIUM_FULL_BUILD=true ./dist-build/android-arm.sh
LIBSODIUM_FULL_BUILD=true ./dist-build/android-armv7-a.sh
LIBSODIUM_FULL_BUILD=true ./dist-build/android-armv8-a.sh
LIBSODIUM_FULL_BUILD=true ./dist-build/android-x86.sh
LIBSODIUM_FULL_BUILD=true ./dist-build/android-x86_64.sh
Copy the code

The output will be here (note that westmere is x86_64).

libsodium has been installed into /Users/asimi/Programming/libsodium-stable/libsodium-android-armv6
libsodium has been installed into /Users/asimi/Programming/libsodium-stable/libsodium-android-armv7-a
libsodium has been installed into /Users/asimi/Programming/libsodium-stable/libsodium-android-armv8-a
libsodium has been installed into /Users/asimi/Programming/libsodium-stable/libsodium-android-i686
libsodium has been installed into /Users/asimi/Programming/libsodium-stable/libsodium-android-westmere
Copy the code

Create a Flutter plugin and bind it to libsodium using FFI.

Let’s create a new empty Flutter plugin.

cd $ROOT_DIR
flutter create --template=plugin flutter_libsodium
Copy the code

Flutter iOS plugin Settings

Copy the surrounding libraries to the correct location.

cp $ROOT_DIR/libsodium-stable/libsodium-ios/lib/libsodium.a $ROOT_DIR/flutter_libsodium/ios/
Copy the code

Update iOS iOS/flutter_libsoap. podspec file to include binary libraries.

diff --git a/ios/flutter_libsodium.podspec b/ios/flutter_libsodium.podspec
index 0ae9b0f.. e4ad522 100644
--- a/ios/flutter_libsodium.podspec
+++ b/ios/flutter_libsodium.podspec
@ @ - 16, 8 + 16, 10 @ @A new flutter plugin project.s.source_files = 'Classes/**/*' s.dependency 'flutter' s.form = :ios, '8.0'+ s.vendored_libraries = 'libsodium.a'# Flutter.framework does not contain a i386 slice. Only x86_64 simulators are supported. s.pod_target_xcconfig = { 'DEFINES_MODULE' => 'YES', 'VALID_ARCHS[SDK =iphonesimulator*]' => 'x86_64'} s.wift_version = '5.0'+  s.xcconfig = { 'OTHER_LDFLAGS' => '-force_load "${PODS_ROOT}/../.symlinks/plugins/flutter_libsodium/ios/libsodium.a"'}
 end
Copy the code

Flutter Android plugin Settings

Copy around the library to the correct location.

mkdir -p $ROOT_DIR/flutter_libsodium/android/src/main/jniLibs/{arm64-v8a,armeabi-v7a,x86,x86_64}
cp $ROOT_DIR/libsodium-stable/libsodium-android-armv7-a/lib/libsodium.so \
    $ROOT_DIR/flutter_libsodium/android/src/main/jniLibs/armeabi-v7a
cp $ROOT_DIR/libsodium-stable/libsodium-android-armv8-a/lib/libsodium.so \
    $ROOT_DIR/flutter_libsodium/android/src/main/jniLibs/arm64-v8a
cp $ROOT_DIR/libsodium-stable/libsodium-android-i686/lib/libsodium.so \
    $ROOT_DIR/flutter_libsodium/android/src/main/jniLibs/x86
cp $ROOT_DIR/libsodium-stable/libsodium-android-westmere/lib/libsodium.so \
    $ROOT_DIR/flutter_libsodium/android/src/main/jniLibs/x86_64
Copy the code

Flutter Dart– Triviality begins

Here is some code to initialize the libsodium library. Before using any part of libsodium, you first need to call sodium_init(). Then let’s practice getting only the version string, which should return “1.0.18”.

Start by adding FFI as a new dependency to pubspec.yaml.

diff --git a/pubspec.yaml b/pubspec.yaml
index 8c63764.. 8247ec0 100644
--- a/pubspec.yaml
+++ b/pubspec.yaml
@ @ 9, 6 + 9, 7 @ @Environment: flutter: "> = 1.10.0" dependencies:+  ffi: ^0.1.3
   flutter:
     sdk: flutter
Copy the code

Then create a new file lib/ libsodium_binding.dart that will contain the first layer that uses FFI to talk directly to the native library.

library bindings;

import 'dart:ffi';
import 'dart:io';

import 'package:ffi/ffi.dart';

final libsodium = _load();

DynamicLibrary _load() {
  if (Platform.isAndroid) {
    return DynamicLibrary.open("libsodium.so");
  } else {
    returnDynamicLibrary.process(); }}// https://doc.libsodium.org/quickstart#boilerplate
// https://github.com/jedisct1/libsodium/blob/2d5b954/src/libsodium/sodium/core.c#L27-L53
typedef NativeInit = Int32 Function(a);typedef Init = int Function(a);final Init sodiumInit = libsodium.lookupFunction<NativeInit, Init>('sodium_init');

// https://github.com/jedisct1/libsodium/blob/927dfe8/src/libsodium/sodium/version.c#L4-L8
typedef NativeVersionString = Pointer<Utf8> Function(a);typedef VersionString = Pointer<Utf8> Function(a);final VersionString sodiumVersionString =
    libsodium.lookupFunction<NativeVersionString, VersionString>('sodium_version_string');
Copy the code

I borrowed this style of binding using typedef’s and lookupFunction from the Dart SDK unit tests. Notice how mechanical and boring these bindings are. This is intentional — it should be possible to automatically generate these findings from libsodium.

Now we create a lib/libsodium_wrapper.dart on top of the binding. It talks to our binding layer, creates convenient wrappers, and ultimately manages memory on our behalf.

import 'package:ffi/ffi.dart';
import 'package:flutter_libsodium/libsodium_bindings.dart' as bindings;

class LibsodiumError extends Error {}

class LibsodiumCouldNotInitError extends LibsodiumError {}

class LibsodiumWrapper {
  LibsodiumWrapper() {
    if (sodiumInit() < 0) {
      throwLibsodiumCouldNotInitError(); }}int sodiumInit() {
    return bindings.sodiumInit();
  }

  String sodiumVersionString() {
    returnUtf8.fromUtf8(bindings.sodiumVersionString()); }}String getSodiumVersionString(final LibsodiumWrapper wrapper) => wrapper.sodiumVersionString();
Copy the code

Creating a wrapper may seem pointless, but when we look at a non-trivial example below, you’ll see why it’s useful. At least it reminds us to call sodium _init() and check its return value.

Note that sodium_version_string is not in malloc memory on the heap, so we do not need to release the return value. I’ll talk more about memory management when we introduce a non-trivial example.

Also note the odd function definition on the last line, because in order to make an asynchronous call using Compute, “the callback argument must be a top-level function, not a closure or an instance or static method of a class.”

Take a look at the Example subfolder in the Flutter_libsodium branch part1 to see how to use the library and test its integration.

  • Dart example/lib/main.dart is the way to use it.
  • Example \test_driver\app_test.dart is an integration test.

To run integration tests, run them as usual.

cd example
flutter drive --target=test_driver/app.dart --android-emulator
Copy the code

The Flutter Dart code is sealed before being dismantled.

This is a complicated, real-world example where you want to encrypt something on the device and then decrypt it on the server. Also, let’s say we want to encrypt data on the device, making it impossible for the device or other adversary to decrypt what it encrypts, or to modify the data without being detected.

The cryptography primitives that give us these primitives are called “sealed”, libsodium calls these sealed boxes “sealed boxes”, This concept comes from Gifford (1981) ‘s “Cryptographic Sealing for Information Secrecy and Authentication”.

First let’s open a new terminal window and generate a public/private key pair on the server side. Start a Python shell.

ipython
Copy the code

The server key pair is then generated.

import base64
from nacl.public import PrivateKey

keypair = PrivateKey.generate()
print("Public key: " + base64.b64encode(keypair.public_key.encode()).decode('utf-8'))
print("Private key: " + base64.b64encode(keypair.encode()).decode('utf-8'))
Copy the code

Results:

Public key: lKSTP8K5YQoHMZOn2+mTLunP3yMgqN1O8GyaqRvHbQE=
Private key: +YownzrW+Bx2dmpQAjuQJr5SEAwd6Bg5NUDHVfKRIY4=
Copy the code

Let’s use the server public key in Flutter to seal the message. There is a lot of boiler template code involved, so be sure to look at the Flutter_libsodium branch part2, especially the seal Box implementation commit, to see all the code. However, there are some important points to note about the branch of part1.

Starting at the top of the binding, notice how we bind to the Crypto_box_SEAL API.

// int crypto_box_seal(unsigned char *c, const unsigned char *m,
// unsigned long long mlen, const unsigned char *pk);
typedef CryptoBoxSeal = int Function(
    Pointer<Uint8> c, Pointer<Uint8> m, int mlen, Pointer<Uint8> pk);
typedef NativeCryptoBoxSeal = Int32 Function(
    Pointer<Uint8> c, Pointer<Uint8> m, Uint64 mlen, Pointer<Uint8> pk);
final CryptoBoxSeal cryptoBoxSeal =
    libsodium.lookupFunction<NativeCryptoBoxSeal, CryptoBoxSeal>('crypto_box_seal');
Copy the code

Unsigned char* is a memory block of C, which in Dart FFI corresponds to Pointer

. These Pointers must point to locally managed memory. But if you start with the Dart string, how do you get a Pointer

in native memory? Here, we use a handy feature of Dart to extend the String object and create a new toUint8Pointer() method, use libsodium’s safe memory allocation sodium_malloc() method, and then copy the original bytes of the String.

extension StringExtensions on String {
  Pointer<Uint8> toUint8Pointer() {
    if (this= =null) {
      return Pointer<Uint8>.fromAddress(0);
    }
    final units = utf8.encode(this);
    final Pointer<Uint8> result = bindings.sodiumMalloc(units.length);
    final Uint8List nativeString = result.asTypedList(units.length);
    nativeString.setAll(0, units);
    returnresult; }}Copy the code

Why am I using libsodium sodium_malloc to allocate memory instead of using the Dart FFI ALLOCATION API? Sodium_malloc is slower but provides the following functions.

  • Protected pages are created before and after memory allocation; If the program accesses the protected page, the application crashes. This provides deep defense against buffer overflows.
  • The allocated memory ismlock()D to avoid it being swapped to disk or becoming part of a memory dump.

These features provide defense in depth, but ultimately you need to avoid assigning Dart objects like Strings to sensitive data such as plaintext; See the “Future jobs and Areas for Improvement” section below.

We add other help extensions to other classes, so we can come up with a wrapper around the underlying crypto_box_SEAL native call. Note that crypto_box_sealBytes is the overhead (32 bytes epitaxially public key, 16 bytes HMAC) that libsodium adds to the encrypted ciphertext.

// https://doc.libsodium.org/public-key_cryptography/sealed_boxes
String cryptoBoxSeal(final String recipientPublicKeyBase64Encoded, final String plaintext) {
  final int cryptoBoxSealBytes = bindings.crypto_box_SEALBYTES();
  final cLength = plaintext.length + cryptoBoxSealBytes;
  final c = bindings.sodiumMalloc(cLength);
  final m = plaintext.toUint8Pointer();
  final Uint8List recipientPublicKey = base64.decode(recipientPublicKeyBase64Encoded);
  final pk = recipientPublicKey.toPointer();
  try {
    bindings.cryptoBoxSeal(c, m, plaintext.length, pk);
    final Uint8List result = c.toList(cLength);
    return base64.encode(result);
  } finally{ bindings.sodiumFree(c); bindings.sodiumFree(m); bindings.sodiumFree(pk); }}Copy the code

Finally, in the actual UI, we want to encrypt some text when the button is pressed. If you perform this computationally intensive call in the UI thread, you will block it and cause jank. Jank means that you block the UI thread for so long that it interferes with the user interface; User input may be ignored, or animation frames may be skipped. So we need to perform this calculation on a different thread. Flutter provides a convenient function compute, but has the limitation that only one argument can be passed to compute, so we create a convenience class to encapsulate the argument.

Future<void> encryptData(final String plaintext) async {
  final encryptedData = await compute(
      cryptoBoxSeal, CryptoBoxSealCall(wrapper, serverPublicKeyBase64Encoded, plaintext));
  setState(() {
    _encryptedData = encryptedData;
  });
}
Copy the code

If you run the Part2 branch of the code, each time you encrypt some data, you’ll get a different ciphertext because libsodium uses a new historical public/private key pair for each call to the sealed box. In a specific operation, when I encrypt foobar, I got 97 j4 + 8 eeefbqhdbgp3a1juofwv zZSCwppjzaneb4f6a1HEWo4GL8RiN8oGILzMaaM8Mz7 / / Z.

Once you have a Base64-encoded hermetically sealed box (that is, encrypted data), imagine that you’ve somehow transferred it to the server. You can then decrypt it on the server.

import base64
from nacl.public import PrivateKey, SealedBox

private_key_encoded = "+YownzrW+Bx2dmpQAjuQJr5SEAwd6Bg5NUDHVfKRIY4="
private_key = PrivateKey(base64.b64decode(private_key_encoded))
unseal_box = SealedBox(private_key)
ciphertext = "zZSCwppjzaneb4f6a1HEWo4GL8RiN8oGILzMaaM8Mz7/97J4+8EEEfbQHDBGp3A1juOFWv/Z"
new_plaintext = unseal_box.decrypt(base64.b64decode(ciphertext)).decode('utf-8')
print(new_plaintext)
Copy the code

This will return to Foobar as scheduled.

Reference code

Take a look at the Flutter_libsodium GitHub repository, especially the tags part1 and part2.

Future work and areas for improvement

For convenience, I skipped the error handling of the crypto_box_SEAL call in Flutter, and the crypto_box_unseal call on the server. Of course, you deal with errors!

When interacting with native code, you need to interact with data that lives somewhere in memory. If you start with memory managed by the Dart runtime, you need to copy it into locally managed memory for local libraries to access it. It’s wasteful. The most efficient memory way to give local libraries access to data is to be careful to make sure you allocate local memory and then use it through views. There is no need to copy from Dart to native, and FFI has given the easy way, so from native to Dart. When working in your own application, consider whether you can use the Pointer

of local memory directly.

Instead of having separate directories bound to libsodium and Flutter, create a separate directory for both directories, view libsodium as a Git submodule, then create a build script to automatically build libsodium and copy its binaries over for easier maintenance. However, I don’t know if the Flutter plugin supports this custom build process.

As discussed in the article, you should be able to parse libsodium’s C header and code automatically to generate FFI.

Translation via www.DeepL.com/Translator (free version)