Michael Yuan, WasmEdge Maintainer
This use from infoQ.com, link: www.infoq.com/articles/we…
Main points of this paper:
- Dapr is a powerful framework for building microservices.
- The WebAssembly VM, such as WasmEdge, provides high-performance and secure runtime for microservice applications.
- Webassembly-based microservices can be written in multiple programming languages, including Rust, C/C++, Swift, and JavaScript.
- WebAssembly programs are embedded in Dapr Sidecar applications and are therefore portable and cross-platform for Dapr host environments.
- The WasmEdge SDK provides a simple way for Tensorflow reasoning to build microservices.
Since its release in 2019, Dapr (Distributed Application Runtime) has quickly become a very popular open source framework for building microservices. It provides building blocks and packaged services commonly used in distributed applications, such as service invocation, state management, message queues, resource bindings and triggers, mTLS secure connections, and service monitoring. Distributed application developers can leverage and use these Web-based apis at Runtime, which are exposed by building blocks. These applications are often called microservices and run as sidecars. Dapr is an example of a multi-Runtime microservices architecture, as described by InfoQ author Bilgin Ibryam.
Dapr’s Sidecar pattern is a lot like a Service Mesh. However, unlike traditional service grids, which are intended to manage applications without any code changes, Dapr applications need to integrate and actively leverage external Dapr building block services.
Microservice applications in Dapr Sidecar can be native client (NaCl) applications compiled in languages like Go and Rust, or managed language applications written in Python or JavaScript. In other words, sidecar applications can have their own language runtime. The Sidecar model allows Dapr to support “anywhere in any language, any framework, anywhere” for its applications.
WebAssembly and WasmEdge
Dapr can run Sidecar applications directly on an operating system or through application containers such as Docker. Containers offer the benefits of portability, ease of deployment, and security, but they also come with significant overhead.
In this article, we present a new way to run Dapr Sidecar applications. We use a simple NaCl written in Rust or Go to listen for API requests to microservices. It passes the request data to the WebAssembly Runtime for processing. The business logic for microservices is WebAssembly functions created and deployed by the application developer.
At the time of publication, you can already handle network requests using the WasmEdge WASI Socket. For details, see github.com/second-stat…
Figure 1. Dapr microservice with WebAssembly functions.
WebAssembly Runtime is ideal for executing business logic functions.
-
WebAssembly programs can run as fast as compiled machine-native binaries and consume fewer resources than containers.
-
WebAssembly supports high-performance languages such as C/C++, Rust, Swift, and Kotlin. It also supports high-level Languages such as JavaScript and DSL (Domain Specific Languages).
-
WebAssembly programs are portable and can be easily deployed across different operating systems and hardware platforms.
-
WebAssembly provides a security sandbox that isolates applications at the Runtime level. Developers can declare security policies to restrict access to the operating system or other resources.
The following table summarizes the pros and cons of different approaches to sidecar applications.
WasmEdge is the leading cloud-native WebAssembly Runtime hosted by CNCF (Cloud Native Computing Foundation) /Linux Foundation. It is one of the fastest WebAssembly Runtime on the market today. WasmEdge supports all standard WebAssembly extensions as well as proprietary extensions such as Tensorflow reasoning, KV storage and image processing, and sockets. Its compiler toolchain supports not only WebAssembly languages such as C/C++, Rust, Swift, Kotlin, and AssemblyScript, but also regular JavaScript.
WasmEdge applications can be embedded in C programs, Go programs, Rust programs, JavaScript programs, or the CLI of an operating system. Runtime is available via Docker tools (e.g. Cri-o), orchestration tools (e.g. K8s), Serverless platforms (e.g. Vercel, Netlify, AWS Lambda, Tencent SCF) and data streaming frameworks such as YoMo and Zenoh.
In this article, I will demonstrate how to use WasmEdge as the Sidecar application Runtime for Dapr.
Quick start
First you need to install the Go, Rust, Dapr, WasmEdge, and RustWASMC compiler tools.
Next, fork or Clone the demo application from Github. You can use this Repo as an application template.
$ git clone https://github.com/second-state/dapr-wasm
Copy the codeThe demo has three Dapr Sidecar applications.
- The Web-port project provides a common Web service for static HTML pages. This is the UI of the application.
- The image-API-RS project provides a WasmEdge microservice that converts input images to grayscale images using grayscale functions. It demonstrates using the Rust SDK with Dapr and WasmEdge.
- The image-API-Go project provides a WasmEdge microservice that uses classification functions to identify and classify objects on input images. It demonstrates the Go SDK’s use of Dapr and WasmEdge.
Figure 2. The Dapr Sidecar microservice in the demo application
You can follow the instructions in the README to start the Sidecar service. Here are the commands to build the WebAssembly function and start the three Sidecar services.
# Build the classify and grayscale WebAssembly functions, and deploy them to the sidecar projects $ cd functions/grayscale $ ./build.sh $ cd .. /.. / $ cd functions/classify $ ./build.sh $ cd .. /.. / # Build and start the web service for the application UI $ cd web-port $ go build $ ./run_web.sh $ cd .. / # Build and start the microservice for image processing (grayscale) $ cd image-api-rs $ cargo build $ ./run_api_rs.sh $ cd .. / # Build and start the microservice for tensorflow-based image classification $ cd image-api-go $ go build --tags "tensorflow image" $ ./run_api_go.sh $ cd .. /Copy the codeFinally, you should be able to see the Web UI in your browser.
Figure 3. Demo application in production environment
Two WebAssembly functions
We have two functions written in Rust and compiled into WebAssembly. They are deployed in the Sidecar microservice to perform the actual work of image processing and classification.
Although our sample WebAssembly function is written in Rust, you can also compile functions written in C/C++, Swift, Kotlin, and AssemblyScript into WebAssembly. WasmEdge also provides support for functions written in JavaScript and DSL.
The Grayscale function is a Rust program that reads the image data from STDIN and writes the Grayscale image to STDOUT.
use image::{ImageFormat, ImageOutputFormat};
use std::io::{self, Read, Write};
fn main() {
let mut buf = Vec::new();
io::stdin().read_to_end(&mut buf).unwrap();
let image_format_detected: ImageFormat = image::guess_format(&buf).unwrap();
let img = image::load_from_memory(&buf).unwrap();
let filtered = img.grayscale();
let mut buf = vec![];
match image_format_detected {
ImageFormat::Gif => {
filtered.write_to(&mut buf, ImageOutputFormat::Gif).unwrap();
}
_ => {
filtered.write_to(&mut buf, ImageOutputFormat::Png).unwrap();
}
};
io::stdout().write_all(&buf).unwrap();
io::stdout().flush().unwrap();
}
Copy the codeWe use RustwasMC to build it and copy it to image-API-RS Sidecar.
$CD functions/ Grayscale $rustUP Override set rustWASmc build --enable-ext $cp./ PKG /grayscale.wasm.. /.. /image-api-rs/libCopy the codeThe classification function is a Rust function that takes as input a byte array of image data and returns a string for classification. It uses the WasmEdge TensorFlow API.
use wasmedge_tensorflow_interface; pub fn infer_internal(image_data: &[u8]) -> String { let model_data: &[u8] = include_bytes! (" models/mobilenet_v1_1. 0 _224 / mobilenet_v1_1. 0 _224_quant. Tflite "); let labels = include_str! (" models/mobilenet_v1_1. 0 _224 / labels_mobilenet_quant_v1_224. TXT "); let flat_img = wasmedge_tensorflow_interface::load_jpg_image_to_rgb8(image_data, 224, 224); let mut session = wasmedge_tensorflow_interface::Session::new( &model_data, wasmedge_tensorflow_interface::ModelType::TensorFlowLite, ); session .add_input("input", &flat_img, &[1, 224, 224, 3]) .run(); let res_vec: Vec<u8> = session.get_output("MobilenetV1/Predictions/Reshape_1"); / /... Map the probabilities in res_vec to text labels in the labels file ... if max_value > 50 { format! ( "It {} a <a href='https://www.google.com/search?q={}'>{}</a> in the picture", confidence.to_string(), class_name, class_name ) } else { format! ("It does not appears to be any food item in the picture.") } }Copy the codeWe use RustwasMC to build it and then copy it to image-apI-go sidecar.
$CD functions/classify $rustup override set $rustWASmc build -- enable_ext $cp./ PKG/classify_bc.wasm.. /.. /image-api-go/lib/classify_bg.wasmCopy the codeIn the next three chapters, we’ll take a closer look at these three Sidecar services.
Image processing sidecar
The image-API-RS Sidecar application is written in Rust. It should already have the WebAssembly function lib/grayscale.wasm installed from the previous step. Install WasmEdge Runtime binary lib/ WasmEdge – Tensorflow-lite and its dependencies by referring to functions/bin/install.sh scripts.
The Sidecar microservice runs a Tokio based event loop that listens for incoming HTTP requests from the path/API /image.
#[tokio::main]
pub async fn run_server(port: u16) {
pretty_env_logger::init();
let home = warp::get().map(warp::reply);
let image = warp::post()
.and(warp::path("api"))
.and(warp::path("image"))
.and(warp::body::bytes())
.map(|bytes: bytes::Bytes| {
let v: Vec<u8> = bytes.iter().map(|&x| x).collect();
let res = image_process(&v);
Ok(Box::new(res))
});
let routes = home.or(image);
let routes = routes.with(warp::cors().allow_any_origin());
let log = warp::log("dapr_wasm");
let routes = routes.with(log);
warp::serve(routes).run((Ipv4Addr::UNSPECIFIED, port)).await
}
Copy the codeOnce it receives the image file in the HTTP POST request, it calls the WebAssembly function in WasmEdge to perform the image processing task. It creates a WasmEdge instance to interact with the WebAssembly program.
pub fn image_process(buf: &Vec<u8>) -> Vec<u8> {
let mut child = Command::new("./lib/wasmedge-tensorflow-lite")
.arg("./lib/grayscale.wasm")
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.spawn()
.expect("failed to execute child");
{
// limited borrow of stdin
let stdin = child.stdin.as_mut().expect("failed to get stdin");
stdin.write_all(buf).expect("failed to write to stdin");
}
let output = child.wait_with_output().expect("failed to wait on child");
output.stdout
}
Copy the codeThe following Dapr CLI command line is used to start the microservice in the Dapr Runtime environment.
$ cd image-api-rs $ sudo dapr run --app-id image-api-rs \ --app-protocol http \ --app-port 9004 \ --dapr-http-port 3502 \ --components-path .. /config \ --log-level debug \ ./target/debug/image-api-rs $ cd .. /Copy the codeTensorflow sidecar
The image-api-go Sidecar application is written in Go. It should already have the WebAssembly function lib/ Classify_bg.wasm installed from the previous step. See functions/bin/install.sh to install WasmEdge Runtime Go SDK.
The Sidecar microservice runs an event loop that listens for incoming HTTP requests from the path/API /image.
func main() { s := daprd.NewService(":9003") if err := s.AddServiceInvocationHandler("/api/image", imageHandlerWASI); err ! = nil { log.Fatalf("error adding invocation handler: %v", err) } if err := s.Start(); err ! = nil && err ! = http.ErrServerClosed { log.Fatalf("error listenning: %v", err) } }Copy the codeOnce it receives the image file in an HTTP POST request, it calls the WebAssembly function in WasmEdge to perform tensorFlow-based image recognition tasks. It uses WasmEdge’s Go API to interact with WebAssembly programs.
func imageHandlerWASI(_ context.Context, in *common.InvocationEvent) (out *common.Content, err error) {
image := in.Data
var conf = wasmedge.NewConfigure(wasmedge.REFERENCE_TYPES)
conf.AddConfig(wasmedge.WASI)
var vm = wasmedge.NewVMWithConfig(conf)
var wasi = vm.GetImportObject(wasmedge.WASI)
wasi.InitWasi(
os.Args[1:], /// The args
os.Environ(), /// The envs
[]string{".:."}, /// The mapping directories
[]string{}, /// The preopens will be empty
)
/// Register WasmEdge-tensorflow and WasmEdge-image
var tfobj = wasmedge.NewTensorflowImportObject()
var tfliteobj = wasmedge.NewTensorflowLiteImportObject()
vm.RegisterImport(tfobj)
vm.RegisterImport(tfliteobj)
var imgobj = wasmedge.NewImageImportObject()
vm.RegisterImport(imgobj)
vm.LoadWasmFile("./lib/classify_bg.wasm")
vm.Validate()
vm.Instantiate()
res, err := vm.ExecuteBindgen("infer", wasmedge.Bindgen_return_array, image)
ans := string(res.([]byte))
vm.Delete()
conf.Delete()
out = &common.Content{
Data: []byte(ans),
ContentType: in.ContentType,
DataTypeURL: in.DataTypeURL,
}
return out, nil
}
Copy the codeThe following Dapr CLI command line is used in the Dapr Runtime environment to start microservices.
$ cd image-api-go $ sudo dapr run --app-id image-api-go \ --app-protocol http \ --app-port 9003 \ --dapr-http-port 3501 \ --log-level debug \ --components-path .. /config \ ./image-api-go $ cd .. /Copy the codeThe web UI sidecar
Web UI service Web-Port is a simple Web server written in Go. It serves static HTML and JavaScript files when it is in a static file, and sends images uploaded to/API/Hello to the Grayscale function or to the/API/Image endpoint of False Sidecar.
func main() { http.HandleFunc("/static/", staticHandler) http.HandleFunc("/api/hello", imageHandler) println("listen to 8080 ..." ) log.Fatal(http.ListenAndServe(":8080", nil)) } func staticHandler(w http.ResponseWriter, r *http.Request) { // ... read and return the contents of HTML CSS and JS files ... } func imageHandler(w http.ResponseWriter, r *http.Request) { // ... . api := r.Header.Get("api") if api == "go" { daprClientSend(body, w) } else { httpClientSend(body, w) } } // Send to the image-api-go sidecar (classify) via the Dapr API func daprClientSend(image []byte, w http.ResponseWriter) { // ... . resp, err := client.InvokeMethodWithContent(ctx, "image-api-go", "/api/image", "post", content) // ... . } // Send to the image-api-rs sidecar (grayscale) via the HTTP API func httpClientSend(image []byte, w http.ResponseWriter) { // ... . The req, err: = HTTP. NewRequest (" POST ", "http://localhost:3502/v1.0/invoke/image-api-rs/method/api/image", bytes.NewBuffer(image)) // ... . }Copy the codeJavaScript in page.js simply upload the image to the/API/Hello endpoint of the Web-port Sidecar, which will request a classification or Grayscale microservice based on the Request Header API.
function runWasm(e) {
const reader = new FileReader();
reader.onload = function (e) {
setLoading(true);
var req = new XMLHttpRequest();
req.open("POST", '/api/hello', true);
req.setRequestHeader('api', getApi());
req.onload = function () {
// ... display results ...
};
const blob = new Blob([e.target.result], {
type: 'application/octet-stream'
});
req.send(blob);
};
console.log(image.file)
reader.readAsArrayBuffer(image.file);
}
Copy the codeThe following Dapr CLI command launches the static UI file for the Web service.
$ cd web-port $ sudo dapr run --app-id go-web-port \ --app-protocol http \ --app-port 8080 \ --dapr-http-port 3500 \ --components-path .. /config \ --log-level debug \ ./web-port $ cd .. /Copy the codeCompleted. You now have a distributed application with three parts, written in two languages.
The following
As we’ve shown, there is a lot of synergy between Dapr’s distributed Network Runtime and WasmEdge’s common language Runtime. This approach can be generalized and applied to other service grids or distributed application frameworks. Unlike Dapr, many service grids can only run on Kubernetes as their control plane and therefore rely on the Kubernetes API. WasmEdge is a Kubernetes-compliant Runtime that can make a difference as a lightweight container alternative for running microservices. Stay tuned!