Let’s contribute to the Add-on ecology of Node.js

The author | Wu Chengzhong (zhao lang)

This post was written by Chengzhong Wu (@Legendecas), Gabriel Schulhof (@GabrielSchulhof), Jim Schlight (@Jimschlight), Kevin Eady, By Michael Dawson (@mhdawson1), Nicola Del Gobbo (@Nicknaso) et al. First published on Node.js Medium.

On N – API

N-api brings an ABI-stable Add-On API to Node.js, simplifying the burden of building and developing add-On support across versions of Node.js.

N-api’s C++ wrapped nod-addon-api is currently being downloaded more than 2.5 million times per week, and all node.js LTS (long term supported versions) now support n-api v3 or higher. Node.js 15.x has also started to support the latest N-API V7. So we thought this would be a good time to take a look back at the current node.js add-on development experience.

When we started working on the N-API in 2016 (the first proposal was made on December 12, 2016), we knew it was going to be a very long-term mission. There are already many existing packages in the Node.js community ecology, so this migration process will continue for quite some time.

But the good news is, we’ve come a long way from where we started. Many of these difficulties have been overcome by several Node.js Collaborator, the N-API team and the module package authors. Currently, n-API has become the default and recommended way to write Node.js add-on.

With the development of N-API, new apis have been added to n-API to meet the new requirements of node. js module package authors to migrate their libraries to N-API. Of course, this process has also maintained the stability and forward compatibility of N-API according to our pre-design.

We are also very pleased to see positive feedback from the module package authors such as twitter.com/mafintosh/s…

Without further discussion, let’s look at some of the new features that have been added to the N-API over the years.

New features

More and more developers are using n-API and Node-addon-API to develop Node.js add-on, and we continue to add new key features and improve the add-on development experience for both n-API and Node-addon-API.

These improvements fall into three main categories, which we describe below.

Multithreaded and asynchronous programming

As the use of Node.js becomes more prominent among developers, the need to work with OS interfaces and asynchronous events is increasing. Node.js is an implementation of JavaScript’s single-threaded model. A Node.js environment will only have one main thread that can access JavaScript values.

Therefore, performing cpu-heavy tasks on the main thread causes the JavaScript program to block, causing events and callbacks to pile up in the event queue. In order to improve the application’s cross-thread data integrity development experience, we gathered a lot of real-world requirements, and both n-api and n-api’s C++ encapsulated nod-addon-api brought various mechanisms to solve the problem of worker threads calling back JavaScript threads. According to the application scenario, it can be divided into:

  • AsyncWorker provides one-way, single callback task encapsulation to notify JavaScript of the final execution result or exception information of the task;
  • AsyncProgressWorker, similar to AsyncWorker, provides one-way, single callback task encapsulation, but adds a mechanism for asynchronously passing progress information to JavaScript;
  • Thread-safe functions provide a mechanism to call back to node.js JavaScript threads from any Thread, any number of threads, and any point in time.

Multiple Node.js context support

One of the most exciting recent features of Node.js is [Worker_Threads], which provides a complete and concurrent node.js JavaScript execution thread independent of the main Node.js JavaScript thread. This also means that node.js add-on can also be loaded and unloaded multiple times in worker threads as workers are started and destroyed.

However, because these worker threads in the same process share the same memory space, multiple add-on instances must consider the possibility of multiple worker threads existing at the same time. In addition, each Node.js process only loads the add-on dynamic library once, which means that thread-unsafe global add-on properties (such as global static variables) can be accessed by multiple threads at the same time and can no longer be stored as crudily.

Similarly, static data members of C++ classes are stored in thread-unsafe ways, so this should be avoided as well. In addition, for add-on, node.js does not guarantee that a single thread will execute only one worker, so thread-local should also be avoided.

In N-API V6, we have introduced a storage space for add-ons for every Node.js instance (main thread JavaScript instance, worker instance, etc.). In this way, add-on can obtain storage space unique to a single Node.js instance in a process. We also provide some helper methods to help add-ons get started with this feature:

  • The NAPI_MODULE_INIT() macro marks add-on as a module that can be loaded and unloaded multiple times by Node.js in the same process.
  • Napi_get_instance_data () and napi_set_instance_data() are used to safely access the globally unique storage space created by a single Node.js instance for the Add-on.
  • Nod-addon-api also provides the addon class, which wraps all of the above methods in a C++ friendly way to give the add-on the storage space it can use in different worker threads. Therefore, add-on developers can store and create add-on data such as global variables through Addon, and Node.js is responsible for creating the space when the current thread uses the add-on.

Other auxiliary functions

In addition to the above important features, we also found a number of types, methods and functions that are commonly used when maintaining node.js add-on, including:

  • The Date object;
  • BigInts.
  • Retrieve any key from a JavaScript object (Symbol, etc.)
  • Remove the underlying storage from the ArrayBuffer created by add-on.

build

The build workflow is an important part of node.js add-on maintainers and add-On users, and is a major focus of the N-API team, such as cmake.js, Nod-pre-gyp, and Prebuild.

Once upon a time, Node.js add-on could only be built using Node-gyp. For some libraries that already use CMake, cmake.js is an attractive option for building add-ons in addition to nod-gyp dependencies. We’ve also released an example of building an add-on using CMake.

Additional details on how to use cmake.js with n-API add-on can be found in the N-API Resource.

An important practical issue after developing a Node.js add-on is that the C/C++ code for the add-on must be compiled and linked locally at NPM install. This compilation process requires a working C/C++ toolchain installed locally. This dependency is often an obstacle to add-On users who do not have these toolchains installed. The current solution to this problem is to pre-build binary packages and then download those pre-built packages directly at installation time.

There are many tools available to pre-build binary packages. Node-pre-gyp typically uploads the built binary packages to AWS S3. Prebuild is similar, but you upload the package to the GitHub Release.

Prebuildify is another option. The advantage of Prebuildify over the above tools is that the binaries are already available locally when NPM install is installed, rather than having to be downloaded from a third-party service. Although the NPM package installed may be larger, in practice the installation process is relatively fast because you don’t need to download it from AWS or GitHub again.

To get started

We already have a number of Nod-addon-examples on GitHub to give developers a quick look at common scenarios for how to use n-API and Nod-addon-API to develop Node.js add-on. The repository’s root directory contains a number of folders that represent different usage scenarios, from a simple Hello World add-on to a complex multithreaded add-on. Each sample directory will contain three subdirectories representing the traditional NAN, n-API, and Node-addon-api development examples. We can immediately start using node-addon-api from the Hello World example by running the following command:

$ git clone https://github.com/nodejs/node-addon-examples.git
$ cd node-addon-examples/1_hello_world/node-addon-api/
$ npm i
$ node .
Copy the code

Another important Resource is the N-API Resource. This site contains information and materials on developing and building Node.js add-on, from beginner to in-depth, such as

  • The tools needed to get started;
  • Migration guidance from NAN to N-API;
  • Comparison of different build systems (Node-gyp, CMake, etc.);
  • Multiple Node.js context support and thread safety.

conclusion

Since its inception, Node.js has supported the ability to expose JavaScript to more features through C/C++ code. Over time, we’ve learned that there have always been many difficulties in implementing, maintaining, and distributing these add-ons. N-api is considered by add-on maintainers to be a very core area to address these difficulties. So the entire N-API team and community set out to build an ABI-stable Add-on API for the Node.js core.

The C apis representing n-API are now part of every Node.js release, and we have a C++ wrapper that provides these C apis with nod-addon-API installed via NPM. N-api was created with the goal of ensuring ABI and API compatibility between different Node.js versions, even Major versions, and this has proven to provide additional benefits:

  • We no longer need to recompile the Add-on module after switching to a larger version of Node.js;
  • We can implement n-API in a runtime environment other than Node.js which uses V8 as a JavaScript engine, which means that add-on developed for Node.js can be compatible with these runtime environments without any code changes, such as Babylon Native, IoT. Js and Electron.
  • N-api is a pure C API, which means we can develop Node.js add-on using languages other than C/C++ at runtime, such as Go or Rust.

N-api has been released since Node.js V8.0.0 as an experimental feature and has been slow to become widely available, but module developers have continued to give us feedback and contributions, which has helped us continue to add new features and develop new tools to help developers build a better add-on ecosystem.

Today, n-API is widely used in add-on development. For example, some of the most used add-on modules have been migrated to n-API based development:

  • Sharp (weekly ~ 900K downloads)
  • Bcrypt (weekly ~ 500K downloads)
  • Sqlite3 (~ 300K downloads per week)

There have been many improvements to the N-API over the past few years. For add-on developers and users, it also provides an experience that is close to native JavaScript modules.

Began to contribute

We are constantly improving the add-on ecology of n-API and Node.js, but we are always in great need of help. There are several ways you can help n-API do better in a variety of scenarios:

  • Migrate your add-on to n-API;
  • Help your application’s dependent add-on migrate to n-API
  • Propose and implement new features for n-API;
  • Proposed and implemented new n-API-based features for Node-addon-API;
  • Fix problems and add test cases for Node-addon-API
  • Fixed problems and added test cases for node-addon-Examples

The original link

This article is ali Cloud original content, shall not be reproduced without permission.