Ecological landscape year-end inventory Rust | the ocean (next)

Author | Zhang Handong

This is the second part of the “2021 Technology Review” series, which focuses on the important progress of Rust ecological map in 2021.

“2021 Year-end Technical Inventory” is a major project launched by Digger, covering Serverless, Service Mesh, big front end, database, artificial intelligence, low code, edge computing and many other technical fields. Looking to the past, to the future, review the development of IT technology in 2021, inventory the major events of IT technology, and forecast the future trend of IT technology. In the meantime, we’re kicking off our 15th tech-themed essay, with a look at what you see as the tech trends of 2022.

The text before

Half a year ago, I wrote “30,000 Words | Rust Industry Research Report 2021”, which consists of an introduction to the Rust language and an inventory of open source applications in the industry. Half a year later, I feel it is necessary to write another year-end Rust ecological survey report. This is because I want to provide you with a comprehensive perspective. By mining various scattered and hidden information on the Internet, I will draw an “ecological map” of Rust so that you can have an objective and fair understanding of the Rust language.

After finishing this report, I came to the conclusion that Rust is not meant to be used to rewrite everything, but to be used to create a new future. Of course, this is only my personal opinion and does not represent anyone or any organization or company. If you have a different point of view, welcome to discuss.

We choose to use Rust, not because it is easy, but because it is hard, because that goal will serve to organize and measure the best of our energies and skills, because that challenge is one that we are willing to accept, one we are unwilling to postpone, and one which we intend to win, and the others, too.

We choose Rust not because it is easy, but because it is difficult. This helps us to organize and measure how much energy and skill we can put into an optimal goal. When we accept a challenge, it means that the challenge is one we are willing to accept, willing to jump in right away, and must win. So should others.

All contents of this report are from public information on the Internet. If there are any errors or contents that are not suitable for this report, please inform us immediately.

The outline

This report contains the following contents:

• Status of the Rust Project
• Status and trends of Rust applications in various fields
• Rust job distribution
• The prevalence of Rust in education

Detailed review

Ecological landscape in the year-end inventory Rust | (last) sea stars, we count the Rust the state of the Project itself And Rust in the operating system, cloud native, Internet, games, and the data processing in the field of 2021 project dynamic that deserves attention.

This article will continue to discuss the application of Rust in other fields, the distribution of Rust jobs and the popularity of Rust language in education.

Application Status and Trend of Rust in various Fields

Machine learning and scientific computing

2021 NLP star project, Transformers

Select Face, a Chatbot startup based in New York, us. The company is a big name in the NLP world, having just raised a $40 million Series B round in March 2021. GitHub has released Transformers, an open source NLP library that integrates Tokenizers.

Tokenizers are Rust’s tokenizers, which enable 1GB of text processing on a select server CPU in less than 20 seconds. One of the bottlenecks in the modern NLP pipeline based on deep learning is tokenization, especially the implementation that is versatile and framework-independent. So, the core of the tokenizer is written in Rust, and there is a Node and Python binding. Provides implementations of today’s most commonly used word dividers, with emphasis on performance and versatility.

Transformer models are widely used by tesla, Google, Microsoft, Facebook and other tech giants. Transformer drives cars in Tesla’s AutoPilot system, completes sentences in Gmail, translates users’ posts on Facebook and answers users’ natural-language queries in Bing.

Arguably, the startup’s use of Rust to build its word segmentation seems to be intrinsically beneficial to both the accuracy and performance of its machine-learning models.

linfa

Linfa is a Rust-implemented Library similar to Python Scikit-Learn designed to provide a comprehensive toolkit for building machine learning applications using Rust. The team also created rust-ML.

Scikit-learn, also known as SkLearn, is an open source Machine learning toolkit based on python. It uses python’s NumPy, SciPy, and Matplotlib libraries for efficient algorithmic applications and covers almost all major machine learning algorithms.

More information: The Book of Rust Machine Learning

The library has released version 0.5 of its Roadmap for 2021, with 20 of the 24 tasks it has accomplished so far. The Roadmap aims to be fully functional with Python’s SciKit-Learn algorithm, which is based on a branch of SciPy developed for machine learning. At present, only Gaussian process, random forest, ensemble learning and minimum Angle regression remain to be completed. This Roadmap has been developed for about two years and should be completed by 2022.

That is, in 2022 there will be a library in the Rust ecosystem that does exactly what Python SkLearn does, but performs better than skLearn.

Polars

On the data manipulation level, everyone likes Pandas’ API. It’s fast, easy and well-documented. But on the production side, Pandas is a bit tricky. Pandas does not extend well… No multi-threading… It’s not thread-safe… It’s not memory efficiency. All this is Rust’s raison d ‘etre.

Polars is a new Dataframe library implemented in Rust with convenient Python bindings. It attempts to read, write, filter, apply functions, group, and merge in a thread-safe manner. It uses Apache Arrow, a data framework specifically built for efficient data processing and data sharing across languages.

Polars has two advantages:

1. It’s a performance killer, see DB-benchmark.
2. Its API is very simple. You don’t have to understand Rust syntax to understand what the interface is doing.

It also has three disadvantages:

1. Building the Dataframe API was difficult, it took Pandas 12 years to get to 1.0.0, and Polars is young, so it’s not mature yet.
2. To use it, you need to be familiar not only with the Polars API, but also with the Arrow API, since Arrow does much of the heavy lifting.
3. The compilation time is too slow, probably around 6 minutes.

Polars is now sponsored by Xomnia. Xomnia is a Dutch artificial intelligence company working on self-driving ships, known as Teslas on the Water.

oxide-enzyme

Oxide-enzyme is another new project created by the Linfa team at the end of 2021. Oxide-enzyme is a Rust front-end library for enzymes that is currently experimental but under active development.

Enzyme is an automatic differential framework proposed by MIT that synthesizes the gradient of a static analyzable program in the form of LLVM intermediate representation (IR). Enzyme can synthesize gradients of any program written for LLVM IR compiler languages, including C, C ++, Fortran, Julia, Rust, Swift, MLIR, etc. to provide native AD functionality for these languages.

The context in which Enzyme appears

Machine learning frameworks such as PyTorch and TensorFlow have become important tools for development. When calculating gradients for algorithms such as back propagation, Bayesian inference, quantization of uncertainty, and probabilistic programming, we need to write all the code into the framework in differential form. This creates problems for introducing machine learning into new domains: in physics simulations, game engines, climate models, the original domain components are not written by the domain-specific language (DSL) of the machine learning framework. Thus rewriting requirements becomes a challenge when introducing machine learning into scientific computing.

To address this, trends include building new DSLS to make the rewriting process easy, or building directly at programming time. These methods allow us to obtain effective gradients, but still need to be rewritten using DSL or differentiable programming languages. To make things easier for developers, researchers from MIT open-source enzymes.

MegFlow

MegFlow is a streaming computing framework for computer vision applications that megvii opened source in September 2021. MegFlow provides a set of visual parsing services for rapid DEPLOYMENT of AI applications. AI application developers can customize features such as invoice scanning and open fire detection in as little as 15 minutes based on MegFlow’s image and video parsing services.

The conventional AI algorithm delivery process is generally divided into four steps: model training, SDK packaging, business integration and delivery acceptance. MegFlow summarizes megmegflow’s years of experience in AI application project implementation, clearly and consistently abstracts concepts such as resource, message and task, and simplifies the long algorithm delivery process into three steps: model training, business integration and delivery acceptance. With MegFlow, AI algorithm researchers can quickly integrate and deploy trained models into easy-to-use AI services and deliver them.

MegFlow’s R&D team investigated a variety of technical solutions, and finally chose Rust asynchronous ecology, which is safe and has no extra cost, to ensure the security and performance of MegFlow.

MegFlow has been open source for four months now and has added a few new features, as well as a new use case: a cat fence. Programmers have created an artificial intelligence cat food machine! , the experimental results are good. Developers can also apply for free computing power from Megvii.

Qdrant: Neural network search engine

Qdrant is a vector similarity search engine. It provides a production-ready service with a convenient API for storing, searching, and managing points (vectors with extra payload). Qdrant is customized for extended filtering support. This makes it useful for all kinds of neural networks or semantically based matching, faceted search, and other applications. Based on Rust implementation, performance is also guaranteed. Embedded neural network encoders can become full-fledged applications for matching, searching, recommending, and so on.

At present

other

Here are a few machine learning frameworks that are still active in maintenance, as of June 2021:

• Tch-rs is a Rust binding of Pytorch’s Cpp API that is currently under active maintenance.
• Ndarray is an open source project developed by Bluss, a senior scientific computing expert on Rust’s official team, which implements matrix and linear arithmetic based on Rust. The goal is to build a scientific computing community in Rust similar to Numpy and OpenBLas. It is the base of machine vision, data mining, biological information and other scientific computing libraries. The main users in the community are universities or research institutes of some related technologies. Huawei is also heavily involved in the library’s development.
• Nalgebra, Rust’s general-purpose linear algebra library, was developed along with Rapier by the Dimforge open source organization.
• Tvm-rs is the Rust binding of TVM, a deep learning automatic code generation method proposed by University of Washington PhD candidate Tianqi Chen and others, which was briefly introduced by Heart of Machine last August.
• Neuronika is a machine learning framework written in Rust, similar to PyTorch, that already implements the most common layer components (dense layer, Dropout layer, etc.) at speeds comparable to PyTorch.
• Whatlang, a natural language recognition project based on Rust implementation.

GUI

GUI project to watch for 2021: SixtyFPS

SixtyFPS is a toolkit that effectively develops a smooth GRAPHICAL user interface for any display: embedded devices, desktop applications, mobile, and the Web (currently compiled to WASM, but only as a demo, SixtyFPS has no intention of using the Web as its main competitive platform). It supports multiple programming languages, currently Rust, C++, or JavaScript. The core library is implemented in Rust, and support for other languages is done through the C-ABI.

Starting in 2021, we’ve seen an increase in the number of startups based on Rust and open source, including the GUI-focused SixtyFPS, which is an open source GUI tool library for all platforms. One of the goals is to replace Qt, in other words, SixtyFPS is basically the next generation QML that most Qt developers have been waiting for.

Both SixtyFPS founders have deep experience in Qt development. One of them worked as a lead developer and maintainer of the QtQml engine at Qt.

Looking at their 2021 summary report, I found that they registered their company in 2021, established a partnership with a consulting firm like tQCS, found their first client and recruited new members. (Feel foreign technology flow entrepreneurship and domestic or a little different). TQCS provides the number one Qt consulting and UI/UX design service in the world. What does it mean to partner with SixtyFPS? That’s a matter of opinion.

In 2022 they will probably have a big embedded support and a new brand name.

Tauri

Tauri is seen as a replacement for Electron, both of which are frameworks for creating desktop applications using Web technologies to develop application user interfaces. Although the two appear to be the same, they are very different.

Here’s a comparison:

Bundle size: Tauri is about 19 times smaller than Electron.

Framework	OS	Bundle size (installer)	Main executable size
Electron 15.3.0	Windows 11 x64	83 MB	137 MB
Tauri 1.0.0 – beta. 10	Windows 11 x64	7MB	4MB
Electron 15.3.0	Linux x64	114 MB (deb)	134 MB
Tauri 1.0.0 – beta. 10	Linux x64	6.5 MB (deb)	16 MB

Memory consumption: Electron is 9 times higher than Tauri.

Framework	OS	Number of processes	Total RAM used
Electron 15.3.0	Windows 11	4	138 MB
Tauri 1.0.0 – beta. 10	Windows 11	8	110 MB
Electron 15.3.0	Linux x64	18	224 MB
Tauri 1.0.0 – beta. 10	Linux x64	1	23 MB

Performance: Tauri is superior

Framework	OS	First launch after build	Second launch
Electron 15.3.0	Windows 11	9940 ms	1420 ms
Tauri 1.0.0 – beta. 10	Windows 11	1230 ms	1120 ms
Electron 15.3.0	Linux x64	2570 ms	1910 ms
Tauri 1.0.0 – beta. 10	Linux x64	1560 ms	1130 ms

However, tauri is more demanding and the Tauri API is not nearly as complete and fleshy as Node.js, so you may need to implement some of the apis yourself using Rust when using Tauri.

The comparison of the above from: githubhelp.com/Fournux/tau… .

In addition, Tauri incubated two more Crate in 2021:

• Wry, a cross-platform WebView library.
• Tao, a cross-platform Window library.

Tauri is currently in beta 1.0 and is not far from a stable release. Tauri chose Rust for security and performance reasons. Tauri promises to provide you with a safe baseline, whether you’re building a Localhost server into your App or doing something else. Other backend languages such as Go, Nim, Python, and Csharp will also be supported in the future, all through the c-API provided by the current Rust kernel.

Egui

Egui is a GUI framework implemented purely in Rust that uses immediate Mode (drawing the DESIRED UI on the screen in real time). The advantages and disadvantages of this immediate mode are explained in the EGUI README, which you should check out if you are choosing a model. In short, the immediate pattern contrasts with the reserved pattern. The main advantage of immediate mode is that the application code is very simple, but the disadvantage is that the layout is more difficult, and the GUI cannot be too complex, otherwise it will burden the CPU.

A Reddit user said they were using Bevy and Egui to build commercial CAD software.

As of now, eGUI has released version 0.16.

other

• Gtk4-rs, another active GUI library is THE Rust Binding for GTK. Gtk4’s Rust Binding gTK4-RS 0.3.1 was released in 2021 and is currently under active maintenance.
• Iced, a GUI framework influenced by the Elm architecture, is based on wGPU. It’s also cross-platform, including the Web UI. 2021 has been under active development, but no new version has been released in the eight months since the release in April 2021. But it is also a promising framework.
• Druid, an experimental Rust native UI toolkit, is worth watching. Druid’s current development has been largely driven by its use in Runebender, a new font editor (in its very early stages).
• Azul is a cross-platform desktop GUI framework. It mixes functional, reactive, and data-oriented programming paradigms with apis for developing cross-platform desktop applications. The two core principles of Azul are not to render invisible objects and to use combinations of DOM trees rather than inheritance. Currently there is a 1.0-alpha release, and the official 1.0 release is not far off.
• Lvgl-rs is the Rust binding of LittlevGL. Littlevgl is a GUI written in pure C with beautiful controls that are easy to transplant from a display interface to a touch control interface.

Web and Web development

Network services

As the world becomes more connected, the size and complexity of network infrastructure software is increasing dramatically. However, the requirements for the software are as strict as ever: not only must it be fast, but it must also be “secure”, meaning able to handle untrusted data without crashing or being vulnerable to security breaches. Traditionally, the two requirements were at odds: Web programmers had to choose a language that provided both speed and security. So Rust is a necessary language for the next generation network infrastructure.

Tokio

Tokio, a mainstay of the current Rust ecosystem for building web services, is a Rust asynchronous runtime and network application framework. While there are other asynchronous runtimes in the Rust ecosystem, such as Async-STD, Tokio is definitely the most ecologically mature one.

Tokio does a lot of work on security, performance, and asynchronous task scheduling, with support for epoll and IO_URING (tokio_uring just released version 0.2). There is also tracing, Console (asynchronous debugging, just released version 0.1), and Loom (concurrent testing) tools.

While Rust asynchrony still needs a lot of work, it is now possible to use Rust asynchrony in a production environment thanks to Tokio.

Lunatic

Lunatic, is a WebAssembly runtime influenced by Erlang. You can use it as a fast, robust, and extensible server-side application, but you can use it from any language that compiles to WebAssembly.

Lunatic concurrency is based on ultra-lightweight processes, similar to Green threads or Go-Routines. The Lunatic processes can be created quickly, occupy small memory, and have low scheduling overhead. They are designed for large-scale concurrency. It is not uncommon to see hundreds of thousands of such processes running simultaneously in a typical application. Lunatic processes are completely isolated from each other, with their own stacks, heaps, and even system calls. If one process fails, it does not affect the rest of the system. This allows developers to create powerful and fault-tolerant abstractions.

All processes running on the Lunatic are scheduled and executed by preemptive asynchronous execution (smoL based implementation). This gives the developer the freedom to write simple blocking code, but the runtime will ensure that it doesn’t actually block the thread while waiting for I/O.

Even if an infinite loop exists somewhere in the code, scheduling will always be fair and will not permanently block the thread of execution. The best part is that developers don’t have to do anything special to achieve this, the runtime handles it no matter which programming language you use.

All of this is based on Rust and WebAssembly (WASmTime and WASmtime-WASI), which has only released version 0.7.0 and is still iterating at a high rate.

Embassy, embedded asynchronous runtime

In embedded, interrupts, DMA, and multiple cores are typically used to process concurrent events without blocking the main loop. These have traditionally been handled by intentional register writes. For example, a command is sent to a peripheral to start a task, the program continues to run, and when the task is complete, an interrupt is triggered and the code that processes the completion runs immediately. These processes can be abstracted in Rust using future-based Async/Await capabilities.

Embassy is a project designed to make Async /await the first choice for embedded development. Asynchrony results in better performance and lower power consumption because the actuator does not have to guess when it will be ready to execute in the future. However, the program size can be larger than other alternatives, which can be a problem for some space-constrained devices with very low memory. On Embassy supported devices, such as STM32 and NRF, memory is usually large enough to accommodate a modest increase in program size.

Embassy is an executor and a hardware access layer (HAL). An executor is a scheduler that typically performs a fixed number of tasks assigned at startup, but more can be added later. HAL is an API that CAN be used to access peripheral devices such as USART, UART, I2C, SPI, CAN, and USB. Embassy provides implementation of asynchronous and blocking apis where it makes sense. DMA (direct memory access) is a good example for asynchrony, while GPIO state is better suited for blocking apis.

Embassy can also provide a system timer that can be used for asynchronous and blocking delays. For less than a microsecond, blocking delays should be used because the cost of context switching is so high that the actuator will not be able to provide accurate timing.

Embassy currently offers HAL for several microcontroller families:

• embassy-nrfNRF microcontroller for Nordic Semiconductor
• embassy-stm32STM32 microcontroller for ST Microelectronics
• embassy-rpSuitable for Raspberry Pi RP2040 microcontroller

The ASYNCHRONOUS trait support in Embassy is an MVP implementation of the Rust Async trait to be officially supported in the future:

pub trait I2c<A: AddressMode = SevenBitAddress> {
    /// Error type
    type Error;
		// Based on GAT (nightly Rust)
    type ReadFuture<'a>: Future<Output = Result<(), Self::Error>> + 'a
    where
        Self: 'a;
		
    fn read<'a> (&'a mut self, addr: A, bs: &'a mut [u8]) -> Self::ReadFuture<'a>;
}

impl<A: AddressMode = SevenBitAddress> I2c<A> for u8{
      fn read<'a> (&'a mut self, addr: u8, bs: &'a mut [u8]) -> Self::ReadFuture<'a> {
          // Use async move
        	async move {
              // implementation}}}Copy the code

When GAT stabilizes (projected in Q1 2022), Rust Embedded asynchronous ecosystems will soon be enriched.

EBPF stable support

EBPF is a set of general purpose execution engines that can sandbox programs in the Linux kernel. EBPF provides the general ability to execute specific code efficiently and safely based on system or program events. The users of general ability are not limited to kernel developers.

EBPF is also increasingly playing an important role in observation (tracking, performance tuning, etc.), security and networking. Well-known Internet companies such as Facebook, NetFlix and CloudFlare have widely adopted various programs based on eBPF technology for performance analysis, troubleshooting, load balancing, and anti-ddos attacks. A number of eBPF tools are built into the Facebook machine, according to the information.

The performance of eBPF technology in network technology is more impressive. The combination of BPF technology and XDP (eXpress Data Path) and TC (Traffic Control) can achieve more powerful network functions.

Redbpf is Rust’s repository of tools for building and running BPF/eBPF modules. It contains many components that make eBPF functionality easier to use. Updates are also very active, with version 2.3.0 just released a week ago. Infinyon uses it to route traffic.

Other actively maintained libraries and frameworks

• Monoio, an asynchronous Runtime based on the IO -uring thread-per-core model, was introduced in the previous article. See Design and Implementation of the Rust Asynchronous Runtime for more details.
• Glommio, another asynchronous Runtime based on the IO -uring Thread-to-per-core model from DataDog open source, has started his own business.
• H3, Rust implementation of HTTP3 protocol.
• Quinn, asynchronous friendly QUIC implementation.
• Tonic, is an asynchronous implementation of gRPC client and server.
• Quilkin, described earlier, is a UDP proxy developed by Google Cloud in collaboration with Embark Studios, tailored for high-performance real-time multiplayer games.
• RedisJSON, RedisJSON is a high-performance JSON document store that allows developers to build modern applications. It stores and processes JSON in memory to support response times of millions of operations per second at the sub-millisecond level. Native indexes, queries, and full-text searches for JSON documents allow developers to create secondary indexes to quickly query data.
• Rust-rdkafka, a fully asynchronous Kafka client implementation.
• Smoltcp, a stand-alone, event-driven TCP/IP stack designed for bare-metal real-time systems.
• Rust-libp2p, libp2p network stack.

Eclipse zenoh

As the number of connected devices steadily increases, the industry is experiencing unprecedented heterogeneity in computing, storage, and communication capabilities, as well as new challenges in generating data and the scale of data that must be delivered and used.

In addition, performance, efficiency, and privacy concerns are driving a growing desire to keep data processing as close to the source as possible while facilitating access to remote applications. In other words, we are experiencing a mainstream architecture shift from a cloud-centric model (from cloud to cloud storage, processing, and collection of data) to an edge-centric model (storing and processing data where it makes the most sense for performance, energy efficiency, and security).

Zenoh, which provides zero-overhead Pub/Sub, Store/Query and computation. Integrate dynamic data, in-use data, and static data with computing. It deftly fuses traditional publish/subscribe with geographically dispersed storage, query, and computation while maintaining time and space efficiencies that far exceed those of any mainstream stack.

Zenoh is designed to meet the needs of applications that must handle dynamic data, static data, and computing in a scalable, efficient, and location-transparent data manner.

Zenon has now released a version code-named Aithusa that provides integration of the DDS Plugin and ROS2 for IoT, robotics and autonomous driving.

Web backend

Some people say that using Rust for Web development is a waste of time, but this view is based on the stereotype of “system-level languages.” Rust is outstanding in terms of performance, engineering architecture, and development efficiency, and it just needs a mature framework. In any case, Rust’s ecosystem in Web development is taking shape.

Poems: Bring Rust into the Web ecosystem

Poem is a Rust asynchronous Web framework developed by Chinese. The most exciting features of Poem are:

• Feature rich, comprehensive.
• It’s very simple to use and the documentation is great. See some examples here
• OpenAPI specification generation support out of the box. This is a very important question because there is no popular Web framework in Rust that natively supports OpenAPI specification generation. OpenAPI support brings Rust into the Web ecosystem.
• Support for middleware functionality (similar to express.js). Here’s an example
• Use out of the box with AWS Lambda.
• Compatible withTower ServiceandLayer.This allows you to do fun things here, like this oneexample.

Known open source projects that use Poem:

• Delicate, a distributed task scheduling platform.
• Databend, a data cloud service implemented by Rust.

Use the closed source project poem

• Muse, NetEase’s internal art resource sharing platform of Thunder, is backed by Rust.

• Hikvision’s HiK-ProConnect project is a front-end automation deployment platform based on AWS continuous integration.

SeaORM

SeaORM is an asynchronous dynamic ORM designed to be the Rust version of Active Record, built on SQLX and SeaQuery.

SeaQuery is a query generator that is the foundation of SeaORM for building dynamic SQL queries in Rust, using an ergonomic API to build expressions, queries, and schemas into abstract syntax trees (AST). MySQL, Postgres, and SQLite are all supported behind the same interface. It is similar to the Arel component of Rails’ ActiveRecord ORM framework.

SeaORM is currently under rapid development with release 0.5. SeaORM looks like a great development experience. For more information on SeaORM, see the Rust Ecology Observation | SeaORM: Making a Rust version of ActiveRecord.

Other interesting frameworks

• Axum, tokio’s newly released Web framework, features macro-free, Tower middleware abstraction, and takes full advantage of the Tower ecosystem. The downside is that generics are used too much.
• Perseus, for example, has added plugin system to support I18N, which is based on Fluent. Fluent has been used to translate rust’s official website. It is very convenient.

Big front end

Front-end infrastructure

Deno

Deno is a modern and secure TypeScritp and JavaScript runtime based on V8 and Rust implementations. Promises, Async /await, ES modules and asynchronous iterators are all first-class citizens in Deno.

To ensure that future developments in JavaScript will continue to apply to Deno and server-side JavaScript runtimes, Deno has joined the ECMA International Standards Organization responsible for the JavaScript standard (ECMA-262) and participated in the TC39 Working Group, Luca Casonato will be Deno’s main representative at TC39.

Deno has also partnered with GFX-RS to support WGPU for gPU-accelerated machine learning out of the box. Because of Deno’s modular architecture, the WebGPU implementation is one of the pluggable components.

Deno is currently in high development with version 1.17.3 released to date. Thirteen companies currently use Deno in their technology stack, including Cloudless, Appwrite, and The Lonero Foundation.

parcel && swc

Parcel is a Rust implemented Web application packaging tool for developers with varying experience. It takes advantage of multi-core processing to provide extremely fast speed and requires no configuration. It is implemented based on SWC. Parcel has also recently opened source a new CSS parsing, transformation, and compression tool called parcel- CSS.

SWC, short for Speedy Web Compiler, is an ultra-fast TypeScript/JavaScript Compiler written in Rust. Release updates are very active, with 1.2.179 just released three days ago.

Project Status:parcel / swc(Rust) vsesbuild (Go)

1. The SWC author and core Parcel contributor has now been hired by next.js; Esbuild is the author’s own side project;
2. SWC currently has at least four full-time developers.
3. SWC has more open source activity than EsBuild: SWC’s total PR numbers are over 1600, while Esbuild has over 200.
4. The issues of SWC and ESBuild are similar in number, but the issues of SWC are clearly classified and managed more carefully, and are linked to the release milestones.
5. SWC and ESBuild issues have very high frequency and timely response.
6. Parcel is active with over 2,000 PR and over 4,000 issues.

Trill company

At present, the business side of tiktok applet is also recruiting Rust developers to practice using Rust/C++ on some front-end infrastructure (such as compilers).

WebAssembly front-end framework

Yew

Yew is an advanced Rust framework designed to create multithreaded front-end Web applications using WebAssembly. Component-based, inspired by React and Elm, it is high-performance and supports interaction with JavaScript. It is currently under active development.

Sycamore

Sycamore is a responsive, virtual-dom-free front-end library, also based on Rust and WebAssembly. It features no JavaScript support because it is not needed.

Other frameworks

• Percy, experimental Rust + WASM front-end framework
• Seed, Rust front-end framework based on Elm architecture.
• Sauron, a versatile Web framework and library for building client and/or server side Web applications, is very focused on simplicity. It is suitable for developing Web applications that use progressive rendering.
• MoonZoon, a full-stack Rust framework under development, claims to have no JS/CSS/HTML, etc., and the development progress is slow.

In other areas

Audio and video

Audio processing

• Creek, real-time disk stream IO for audio.
• Symphonia, a library for pure Rust multimedia format demultiplexing, tag reading and audio decoding.
• Cpal, pure Rust’s cross-platform audio I/O library.
• Dasp, for digital audio signal processing.
• Meadowlark aims to be a free and open source DAW (digital Audio Workstation) for Linux, Mac and Windows.

Video processing

• Rsmpeg is a secure FFmpeg Rust binding library.
• Gstreamer, the Rust binding library for Gstreamer. GStreamer is a pipeline-based multimedia framework, based on GObject and written in C. Pipelined design allows you to create many multimedia applications such as video editors, streaming media broadcasts, and media players.
• Rav1e, the fastest and safest AV1 encoder.

Audio and video media services

• Xiu, a simple and secure pure RUST (RTMP/HTTPFLV/HLS /relay) real-time media server.
• Signal-calling-service, Selective Forwarding Units (SFU) based on WebRTC, which implements GOOGCC and transport-CC congestion control in the form of flow processing. This is open-source by Signal, a cross-platform encrypted messaging service.
• Webrtc-rs, a pure Rust implementation of the WeBRTC stack, which overrides the Pion stack with Rust. The library is also under active development in 2021. Most of the features in the roadmap are already complete.

Rust and blockchain

Rust and smart contracts

Many blockchains either run smart contracts written in Rust or implement their smart contract runtime or language in Rust. They fall into several categories:

1. Based on WebAssembly smart contracts, Rust can be compiled into WASM, such as Elrond, Holochain, NEAR, Secret Network, Substrate. Most of these have the Rust SDK.
2. Tools that support compiling Rust to other VMS, for exampleNervos(RISC – V) andSolana(eBPF).
   1. The Nervos contract is written using Rust and their Capsule library. In addition to being the only blockchain running RISC-V, Nervos is a rare smart contract blockchain that uses a UTXO model instead of an account model.
   2. Solana runs a particularly odd VM: a user-space VM RBPF based on eBPF. Solana has put together an effective Rust->eBPF tool chain, which includes a branch of the Rust compiler that supports eBPF. Of course, since eBPF does not support loops, Rust here must be interesting.
3. People implementing smart contract languages in Rust, such as Solang (a Solidity to WASM compiler), Leo (a zero-knowledge language), and Move (a language for Diem). While many chains are moving to general-purpose VMS, and WASM in particular, there are two main reasons not to do so: a desire to be compatible with Ethereum EVM, and a proof based on zero knowledge. Move is a relatively rare case in that it is both its own language and its own VM, but there is no obvious reason to have a custom VM like a zero-knowledge language.

Some noteworthy projects

• solang, is a Solidity compiler implemented with Rust but not for Ethereum EVM. Instead, it uses LLVM as the back end, which means it can potentially target the majorityDon’tThe blockchain that runs EVM. Currently including Substrate, Solana,ewasmandSawtooth.
• Rust-umbral, is a proxy reencryption scheme in which Alice, the owner of the ciphertext, can designate a third party (the agent) to reencrypt the ciphertext to be decoded by Bob without disclosing the plaintext to the agent. It was developed by the NuCypher Project.
• Noir, is a language and compiler for building zero-knowledge proofs. It is intended for use with multiple proofs generating backends, the first of which is for Barretenberg, as used by the Aztec Network.
• MASQ, is a mesh overlay network, somewhat like TOR, but with crypto economic incentives to participate.
• TDN, a trusted Distributed network, is a framework for building distributed applications that support P2P communication, multiple cross-communication blockchains, multi-layer blockchains, and multiple account models. It is built on Chamomile, a new P2P library.
• bip32.This is BIP32Hierarchical deterministic Wallet and BIP39 mnemonic pure Rust, universal,no_stdFriendly implementation.
• Jsonrpsee, an asynchronous implementation of JSON-RPC from Parity, which also created JsonRPC Crate.
• Stateright is a model checker for implementing distributed systems in Rust. Unlike traditional model checker such as TLA+, Stateright is the Rust DSL that checks the Rust production implementation of the system. It includes examples of Paxos and other simple algorithms.
• Arti, which is Rust’s official rewrite of Tor, was first announced in July. Tor is a key part of the distributed infrastructure, and this project enables Tor to be embedded in our applications.
• Openmls, the Rust implementation of the Messaging Layer Security protocol, is an ongoing standard being developed by the IETF MLS Working Group. The e protocol is designed to address a problem still prevalent in secure messaging: while strong end-to-end encryption for two-party conversations is now possible, there are few good solutions for end-to-end encrypted group communications. MLS specifies “a key establishment protocol that provides efficient asynchronous group key establishment for groups ranging in size from 2 to thousands, with forward secrecy and post-disclosure security”.
• Mina-rs is an implementation of MINA in Rust developed by ChainSafe. It was developed not only with the mobile environment in mind, but also with WASM in mind, indicating that we will be able to embed an entire node directly into the browser. Mina is a new blockchain network that uses zero-knowledge proofs to verify the state of the chain without having to access the entire blockchain, just a tiny (~ 22K) proof.
• Arloader is a command line client for uploading files to Arweave. Files posted to Arweave will be stored permanently. This platform is typically used to store NFT information. Fees can be paid using Arweave native tokens or SOL.

Blockchain ecological information

In 2021, Near/Solana/ Dfinity/NeverOS will be the most popular blockchain public chain, with developer incentive programs and hackathons throughout the year.

More active concepts in 2021 are cross-chain, DeFI, NFT and Web3.

But blockchain is such a big ecosystem that a long article could be written in its own right. But thanks to sites like Rust in Blockchain ❤ Rib. rs, we’re keeping track of the Blockchain ecosystem on a monthly basis. If you are interested, you can look it up yourself.

Yuan universe

The metasverse is arguably the craziest concept of 2021, but its tech stack is very relevant to graphics, artificial intelligence, blockchain, VR/AR, IoT, and more. So is Rust ready on these fronts? Among them, the fields of graphics, artificial intelligence, blockchain and IoT have been inventorized before, and the specific status has been clear.

VR/AR, currently Rust ecology is also done, but not very rich:

Openxrs, which is a Rust binding to OpenXR. OpenXR is an application Programming Interface (API) for XR applications. It is an open source standard that developers can use OpenXR to build and deploy applications across devices.

XR refers to the continuum of real and virtual combined environments generated by computers through human-computer interaction, including technologies related to virtual reality (VR), augmented reality (AR) and mixed reality (MR). OpenXR is an interface between an application and an in-process or out-of-process “XR runtime system,” or simply “runtime.” The runtime can handle functions such as frame composition, peripheral management, and raw trace information.

Rust’s support for VR/AR may have started with OpenXR.

Makepad, is a VR, Web and native rendering UI framework and IDE, based on Rust and WebAssembly (WebGL) technologies. The writer is the founder of Cloud9 IDE. The project also contains a white paper that sets out its vision. The current development progress is not very frequent.

Information security

A quick list of some security tools or frameworks that are currently being actively maintained:

• rustscan, is a modern high-performance port scanner, and provides a scripting engine to support running Python, Lua, Shell. Adaptive learning is supported. The more you use it, the smarter RustScan gets, not based on bloated machine learning, but on basic math.
• feroxbuster, a high performance content discovery tool for penetration testing. Feroxbuster uses brute force combined with word lists to search for unlinked content in target directories, which is an attack. These resources may store sensitive information about Web applications and operating systems, such as source code, credentials, internal network addressing, and so on. This attack is also known as predictable resource location, file enumeration, directory enumeration, and resource enumeration.
• Enarx, an organization led by Red Hat and other well-known companies to create application deployment systems for trusted execution environments.
• sn0int, semi-automatic Advanced Open Source Intelligence (OSINT) framework and package manager for scanning target IP addresses, email, web sites and organizational information and gathering intelligence information from different message sources.
• sniffglueMultithreaded network sniffer. Kpcyrd used tcpdump a lot, but he thought it would be nice if the output was friendlier, and wireshark was often bug-prone, as was tcpdump, so he implemented it in Rust.
• ripasso, a password management tool.
• Rustpad, a multithreaded successor to the classic Padbuster, is written in Rust. It uses the Padding Oracle vulnerability to decrypt any ciphertext or encrypt any plain text without knowing the encryption key!
• Vaultwarden, an unofficial Bitwarden (password management) compatible server, formerly known as Bitwarden_RS.
• Innernet, a private networking system that uses WireGuard (next-generation VPN technology) in the background.
• Moonwalk, during Linux Exploitation, covers traces by leaving zero traces in the system log and file system timestamp. To assist red Team operations only.
• CaptfEncoder, CaptfEncoder is a cross-platform network security tool suite, providing network security related coding conversion, classical cryptography, cryptography, public key encryption, aggregation query, and a variety of utilities. CaptfEncoder V3 uses Rust to develop executable programs that are smaller, faster, better performing, and more powerful. Its author is visually detected in Aliyun.

Scientific research and art

• Nannou aims to let artists create their own art. Mindbuffer, a German firm, is based on Nannou and Koto to create physical art projects: using 486 steper motors, 86,000 leds and a five-channel particle synthesis engine, it creates colorful electronic art that can change shapes.
• Glicol, a graph-oriented real-time music programming language. Glicol helps you make music using code. You can simply connect different nodes to form loops, written in Rust, which, thanks to WebAssembly, runs silkily in a browser.
• 3division-Rust, a physical science researcher, uses RUST to simulate three photons, constructing an exotic form of “molecular” light.
• Varlociraptor implements a novel, unified approach to complete uncertainty perception for invoking genomic variation in arbitrary scenarios. This repository is a bioinformation processing repository mentioned in the article why Scientists Turn to Rust. It is still under active maintenance.
• Eigenvalues, a library of eigenvalue solvers implemented by researchers at the Centre for Electronic Technology in the Netherlands, are described in the article applying Rust to Scientific Numerical Applications: Learning from Past experiences.

Rust occupational post

Occupational classification

Due to the security nature of Rust, there are many companies applying Rust in the financial field, so the largest distribution of Rust jobs in the world is blockchain and quantitative finance.

Basically, according to the current global Rust job recruitment, it can be divided into the following categories:

1. Blockchain/quantitative finance/banking
2. Infrastructure (cloud native platform development) : database/storage/data service/operating system/container/distributed system
3. Platform tools: Remote desktop/remote service products/SaaS/remote work products (such as Nexthink)
4. AI/machine learning/robotics
5. Client cross-platform component development
6. Security Engineer: Blockchain security/information security
7. Embedded engineer
8. Advertising services, for exampleAdinmo
9. Audio and video real-time communication engineer
10. Electric business platform
11. Software consulting

Specific Rust job listings can be found in Rust Weekly/Reddit R/Rust Channel/Rust Magazine/Rustcc forums, as well as on various job sites.

Part of Rust job information in China

1. Bytedance. Rust is used for bytedance’s internal infrastructure, Feishu and Douyin. There are many Rust job requirements.
2. Non-convex technology. Domestic enterprises to do quantitative, futures/stock market.
3. Sea star map. Rust is a subsidiary of Haizhi Group. Can be remotely.
4. Datan Technologies. A startup led by serial entrepreneur Dr. Pu Wang that uses Rust for distributed storage. Can be remotely.
5. Databend. A startup using Rust as a data cloud. Can be remotely.

There are a lot of other jobs out there, mostly blockchain positions, which I won’t list here.

Complete global inventory of Remote Rust jobs

Please note that some of the jobs claimed to be completely remote are actually only considered for US/Canada and sometimes even EU residents, depending on your luck and communication skills!

• RedHat, Rust position, remote. You can find out more at Reddit @sheepdog69.
• Tangram, a developer of machine-learning tools, can send email inquiries about working remotely, but the position is only open to U.S. / Canadian/European candidates.
• The Materialize real-time Streaming database (described earlier) has remote positions, which require email inquiries.
• Toyotaconnected, currently only allowed for remote travel in 12 U.S. states, is a member of the Rust Foundation. They use Rust for machine learning jobs and devices. A lot of interoperability in vendor C and CPP code and a lot of cool, interesting issues that we’re solving inside Toyota and Lexus cars as 5G becomes more and more popular.
• Era Software, Remote (US/Canada/Europe), competes with Elasticsearch. All database engineering jobs are for Rust.
• Qovery, Remote (EU), Qovery is a platform that gives any developer the super ability to deploy their applications to the cloud in seconds.
• Estuary, remote Rust developers to develop Flow, our real-time data platform, will need email inquiries.
• Infinyon, Remote (US), for building a reliable and scalable infrastructure for real-time data streaming, with Fluvio and Infinyon Cloud products.
• Ockam, Ockam is building open source, end-to-end encryption and mutual verification communication protocols and libraries that work in resource-constrained, intermittently connected iot devices and connected machines.
• 1passwordRemote (,Canada, the US and the UK), password management software.
• IOHK, remote, blockchain, for details please click the link to apply for a job.

The above list is only a part of the visible network, for domestic partners, these positions may not apply. For foreign partners, may be a blessing.

The prevalence of Rust language education

Published books at home and abroad

A list of books published at home and abroad from 2018 to 2022.

domestic

• The Simple Truth of Rust
• The Rust Way of Programming
• The Authoritative Guide to Rust
• Rust Programming: Getting Started, Practicing, and Advancing

Abroad:

• Rust in Action
• Programming Rust, 2nd Edition
• Rust for Rustaceans
• Refactoring to Rust
• Black Hat Rust
• Hands-on Rust
• Zero To Production In Rust

These books are not all, there are many foreign books not listed.

It can be seen that the number of Rust books is increasing in foreign countries and also in China.

Popularization of higher education

domestic

• As far as I know, Rust is only applied in the OS courses of Tsinghua University in China. Also see: operating system written in Rust | tsinghua rCore OS tutorial.

Abroad:

• Warsaw University, Poland, has decided to start Rust for second-year students.
• Professor Lin Zhong of Yale University has published “CPSC 429A, Principles of Computer System Design” and “CPSC 425B, Mobile and Embedded Systems” which support Rust. But the course is not open.

Reddit discussion: What would your dream Rust class look like?

A teaching assistant at Warsaw University in Poland started the discussion on Reddit and has been teaching memory management and programming in C (first year) and concurrent and parallel programming in Java and C (second year). Now the school has decided to start Rust for second-year students, and he has been entrusted with preparing the lessons and teaching them. He wants to ask Rust developers for advice. What would your dream Rust University curriculum look like?

His current teaching plan goes something like this:

• Use the official The Book as a textbook. As a teacher, he will condense the knowledge in the book into PPT for teaching.
• In the process of learning grammar, there are also some best practices of Rust ecology.
• After learning the book, I will focus on asynchrony.
• Set up five small tasks of different levels for students to complete, each task has ten days to complete.

See the Reddit post for more details. In the comments, someone offered some advice:

• An undergraduate using Rust to complete the Reinforcement library: github.com/ZuseZ4/Rust… Rust is used in the teaching of machine learning because it is easier to learn machine learning using Python. (It is also reasonable to say that when learning, pay attention to the learning goal, the learning goal of students is to learn the field of machine learning, not Rust).
• A friend said that students should be taught how to solve problems in college. For example, by learning the realization of multi-threaded life game, to learn to solve the problem of multi-threaded concurrency.

What do you think about that?

conclusion

It is very tiring to conduct ecological research on Rust, because Rust, as a truly universal language, has too much ecological information and involves many fields.

But I also insisted on finishing this report because I really like The Rust community, which I see as a community full of hope and vitality.

I hope you like Rust as much as I do.

Related links:

Serverless: industry, academic, community blossomed everywhere, domestic manufacturers quickly stuck

Kubernetes Ecology: large version “inside volume”, safety is worth paying attention to

Big front End: The front end is in deep water, and low code for developers continues to heat up

Year-end inventory service grid: Practicality first, ecology first