Node-ffi Usage Guide

In NodeJS/Elctron, you can call the dynamic link library through Node-FFi and Foreign Function Interface, commonly known as DLL, to realize the call C/C++ code, so as to realize many node functions that are not well realized, or reuse many functions that have been realized.

Node-ffi is a Node.js plug-in for loading and calling dynamic libraries using pure JavaScript. It can be used to create bindings to local DLL libraries without writing any C ++ code. It also handles type conversions across JavaScript and C.

This method has the following advantages over Node.js Addons:

1. No source code is required. 2. There is no need to recompile 'node' every time, the '. Node 'referenced by' node.js' Addons will have a file lock, which will cause trouble applying hot updates to 'electron'. 3. Developers are not required to write C code, but some knowledge of C is still required.Copy the code

The disadvantage is that:

2. Similar to FFI debugging in other languages, this method is almost called in a black box, and errors are difficult.Copy the code

The installation

Node-ffi implements memory sharing between C code and JS code through Buffer class, while type conversion is implemented through ref, ref-array and ref-struct. Since Node-ffi /ref contains C native code, installation requires configuring the Node native plug-in compilation environment.

// Run bash/ CMD /powershell. Otherwise, insufficient permissions will be displayed
npm install --global --production windows-build-tools
npm install -g node-gyp
Copy the code

Install libraries as required

npm install ffi
npm install ref
npm install ref-array
npm install ref-struct
Copy the code

If the project is a electron project, the electron rebuild plug-in can be installed to easily traverse all libraries that need rebuild in node-modules and recompile them.

npm install electron-rebuild
Copy the code

Configure the shortcut in package.json

package.json "scripts": { "rebuild": "cd ./node_modules/.bin && electron-rebuild --force --module-dir=.. /.. / "}Copy the code

Then run the NPM run rebuild operation to recompile the ELECTRON.

Simple example

extern "C" int __declspec(dllexport)My_Test(char *a, int b, int c);
extern "C" void __declspec(dllexport)My_Hello(char *a, int b, int c);
Copy the code

import ffi from 'ffi'
// 'ffi.Library' is used to register functions. The first entry parameter is the DLL path, preferably the absolute file path
const dll = ffi.Library( './test.dll', {
    // My_Test is a function defined in the DLL. The two names must be the same
    // [a, [b, c....]] A is the input parameter type of the function, and [b, c] is the input parameter type of the DLL function
    My_Test: ['int'['string'.'int'.'int']], // You can use text to represent types
    My_Hello: [ref.types.void, ['string', ref.types.int, ref.types.int]] // 'ref.types. Xx' is preferred for type checking, and special abbreviations for 'char*' are explained below
})

// Synchronous call
const result = dll.My_Test('hello'.3.2)

// Asynchronous invocation
dll.My_Test.async('hello'.3.2, (err, result) => {
    if(err) {
        //todo
    }
    return result
})
Copy the code

Variable types

There are four basic data types in C: —- integer floating-point pointer aggregation types

basis

Integer and character types are both signed and unsigned.

type	The minimum range
char	0 ~ 127
signed char	– 127 ~ 127
unsigned char	0 ~ 256

Defaults to signed when unsigned is not declared

Ref and unsigned will be abbreviated to u, as in uchar and the unsigned char.

We have float double long double.

The ref library has already prepared the mappings for the base types.

C + + type	Ref corresponding type
void	ref.types.void
int8	ref.types.int8
uint8	ref.types.uint8
int16	ref.types.int16
uint16	ref.types.uint16
float	ref.types.float
double	ref.types.double
bool	ref.types.bool
char	ref.types.char
uchar	ref.types.uchar
short	ref.types.short
ushort	ref.types.ushort
int	ref.types.int
uint	ref.types.uint
long	ref.types.long
ulong	ref.types.ulong
DWORD	ref.types.ulong

DWORD is a WinAPI type, which is described in detail below

For more extensions, go to Ref Doc

In ffi.Library, types can be declared either in ref.types. XXX or in text (as uint16).

character

The character type is composed of char. In GBK encoding, a Chinese character occupies two bytes, and in UTF-8 it occupies three to four bytes. A ref.types. Char defaults to one byte. Create memory space that is long enough for the required character length. In this case, the ref-array library is required.

const ref = require('ref')
const refArray = require('ref-array')

const CharArray100 = refArray(ref.types.char, 100) // declare the char[100] type CharArray100
const bufferValue = Buffer.from('Hello World') // Hello World converts Buffer
// Iterate through Buffer
const value1 = new CharArray100()
for (let i = 0, l = bufferValue.length; i < l; i++) {
    value1[i] = bufferValue[i]
}
// Initialize the type with ref. Alloc
const strArray = [...bufferValue] // Need to convert 'Buffer' to 'Array'
const value2 = ref.alloc(CharArray100, strArray)
Copy the code

When passing Chinese character types, we must know the encoding mode of DLL library in advance. Node uses UTF-8 encoding by default. If the DLL is not utF-8 encoded, transcoding is required. Iconv-lite is recommended

npm install iconv-lite
Copy the code

const iconv = require('iconv-lite')
const cstr = iconv.encode(str, 'gbk')
Copy the code

Attention! After using encode transcoding, CSTR is the Buffer class, which can be directly used as uchar type

Iconv. Encode GBK (STR. ‘GBK’) in the default use unsigned char | 0 ~ 256 store. If need C code is signed char | – 127 ~ 127, you may need to use the data from the buffer int8 type conversion.

const Cstring100 = refArray(ref.types.char, 100)
const cString = new Cstring100()
const uCstr = iconv.encode('Agricultural Pill'.'gbk')
for (let i = 0; i < uCstr.length; i++) {
    cString[i] = uCstr.readInt8(i)
}
Copy the code

Char []/char * returns the value set by the C code for the character array char[]/char *. If it is a pre-initialized value, it usually ends with a long string of 0x00, which requires manual trimEnd. If it is not a pre-initialized value, it ends with an undefined value, which requires C code to explicitly return the length of the string array returnValueLength.

Built-in shorthand

There are some abbreviations built into FFI

ref.types.int => 'int'
ref.refType('int') = >'int*'
char* => 'string'
Copy the code

Only ‘string’ is recommended.

Strings are considered basic types in JS, but are represented as objects in C, so they are considered reference types. So string is actually char*, not char

Aggregate type

Multidimensional array

A primitive type defined as a multidimensional array needs to be created using ref-array

    char cName[50] [100] // Create a cName variable to store 50 names with a maximum length of 100
Copy the code

    const ref = require('ref')
    const refArray = require('ref-array')

    const CName = refArray(refArray(ref.types.char, 100), 50)
    const cName = new CName()
Copy the code

The structure of the body

Structs are common types in C and are created using ref-struct

typedef struct {
    char cTMycher[100];
    int iAge[50];
    char cName[50] [100];
    int iNo;
} Class;

typedef struct {
    Class class[4].
} Grade;
Copy the code

const ref = require('ref') const Struct = require('ref-struct') const refArray = require('ref-array') const Class = Struct({// note that 'Class' returned is a type cTMycher: RefArray(ref.types. Char, 100), iAge: RefArray(ref.types. Int, 50), cName: RefArray(RefArray(ref.types. Char, 100), 50)}) const Grade = Struct({ RefArray(Class, 4)}) const grade3 = new Grade() // Create an instanceCopy the code

Pointer to the

A pointer is a variable whose value is the address of the actual variable, that is, the direct address of the memory location, somewhat similar to a reference object in JS.

C uses * to represent Pointers

For example, int a* is a pointer to the integer a variable. & is used to fetch the address

int a=10.int *p; // define a pointer to an integer type 'p'
p=&a // assign the address of variable 'a' to 'p', that is, 'p' points to 'a'
Copy the code

The principle of node-ffi pointer implementation is to use the Buffer class to realize memory sharing between C code and JS code with the help of REF, making Buffer become a pointer in C language. Note that once ref is referenced, the prototype of Buffer is modified, and some methods are replaced and injected. Refer to the ref documentation

const buf = new Buffer(4) // Initialize an untyped pointer
buf.writeInt32LE(12345.0) // Write the value 12345

console.log(buf.hexAddress()) // Get the hexAddress

buf.type = ref.types.int // Set the type of BUf corresponding to C, you can modify 'type' to implement C casting
console.log(buf.deref()) // deref() gets the value 12345

const pointer = buf.ref() // Get a pointer to a pointer of type 'int **'

console.log(pointer.deref().deref())  // deref() gets the value 12345 twice
Copy the code

I want to clarify the two concepts a structure type and a pointer type, and I want to do that in code.

Declare an instance of a class
const grade3 = new Grade() // Grade is the structure type
// The structure type corresponds to the pointer type
const GradePointer = ref.refType(Grade) // The structure type 'Grade' corresponds to the type of the pointer that points to Grade
// Get a pointer instance to grade3
const grade3Pointer = grade3.ref()
// deref() gets the value of the pointer instance
console.log(grade3 === grade3Pointer.deref())  // It is not the same object in the JS layer
console.log(grade3['ref.buffer'].hexAddress() === grade3Pointer.deref()['ref.buffer'].hexAddress()) // But it actually refers to the same memory address, i.e. the values referenced are the same
Copy the code

By ref. Alloc (Object | String type,? Value) → Buffer gets a reference object directly

const iAgePointer = ref.alloc(ref.types.int, 18) // Initialize a pointer to class 'int' with a value of 18
const grade3Pointer = ref.alloc(Grade) // Initialize a pointer to the 'Grade' class
Copy the code

The callback function

C’s callback function is normally passed in as an input parameter.

const ref = require('ref')
const ffi = require('ffi')

const testDLL = ffi.Library('./testDLL', {
    setCallback: ['int', [
        ffi.Function(ref.types.void,  // ffi.Function declares the type. 'pointer' also declares the type
        [ref.types.int, ref.types.CString])]]
})


const uiInfocallback = ffi.Callback(ref.types.void, // ffi.callback returns an instance of the function
    [ref.types.int, ref.types.CString],
    (resultCount, resultText) => {
        console.log(resultCount)
        console.log(resultText)
    },
)

const result = testDLL.uiInfocallback(uiInfocallback)
Copy the code

Attention! If your CallBack is called from a setTimeout, there may be a GC BUG

process.on('exit', () => {
    /* eslint-disable-next-line */
    uiInfocallback // keep reference avoid gc
})
Copy the code

The code examples

Here’s a full quote example

/ / header files
#pragma  once

//#include ".. /include/MacroDef.h"
#define	CertMaxNumber 10
typedef struct {
	int length[CertMaxNumber];
	char CertGroundId[CertMaxNumber][2];
	char CertDate[CertMaxNumber][2048];
}  CertGroud;

#define DLL_SAMPLE_API  __declspec(dllexport)

extern "C"{

// Read the certificate
DLL_SAMPLE_API  int My_ReadCert(char *pwd, CertGroud *data,int *iCertNumber);
}
Copy the code

const CertGroud = Struct({
    certLen: RefArray(ref.types.int, 10),
    certId: RefArray(RefArray(ref.types.char, 2), 10),
    certData: RefArray(RefArray(ref.types.char, 2048), 10),
    curCrtID: RefArray(RefArray(ref.types.char, 12), 10})),const dll = ffi.Library(path.join(staticPath, '/key.dll'), {
    My_ReadCert: ['int'['string', ref.refType(CertGroud), ref.refType(ref.types.int)]],
})

async function readCert({ ukeyPassword, certNum }) {
    return new Promise(async (resolve) => {
        // ukeyPassword is a string. The value of C is char*
        ukeyPassword = ukeyPassword.toString()
        // Create a new memory space based on the structure type
        const certInfo = new CertGroud()
        // create an int 4 byte memory space
        const _certNum = ref.alloc(ref.types.int)
        // certinfo.ref () is passed as a pointer to certInfo
        dll.My_ucRMydCert.async(ukeyPassword, certInfo.ref(), _certNum, () => {
            // Clear invalid empty fields
            let cert = bufferTrim.trimEnd(new Buffer(certInfo.certData[certNum]))
            cert = cert.toString('binary')
            resolve(cert)
        })
    })
}
Copy the code

Common mistakes

• Dynamic Linking Error: Win32 error 126

There are three reasons for this error

1. The DLL file cannot be found. You are advised to use the absolute path.
2. If it’s on X64node/electronThis error is also reported for DLLS that reference 32 bits, and vice versa. Make sure the DLL requires the same CPU architecture as your runtime environment.
3. DLL also references other DLL files, but cannot find the referenced DLL file, may be the VC dependent library or there is a dependency relationship between multiple DLLS.

• Dynamic Linking Error: Win32 Error 127: A function with the corresponding name is not found in the DLL. Check whether the function name defined in the header file is the same as the function name written when the DLL is called.

The Path is set

A Dynamic Linking Error will occur if you have multiple DLLS calling each other. This is because the default process Path is the directory where the binary file resides, that is, the node.exe/electron. Exe directory is not the directory where the DLL resides. You can solve the problem as follows:

// Call winAPI SetDllDirectoryA to set the directory
const ffi = require('ffi')

const kernel32 = ffi.Library("kernel32", {
'SetDllDirectoryA': ["bool"["string"]]
})
kernel32.SetDllDirectoryA("pathToAdd")

// Method 2 (recommended) : Set Path environment
process.env.PATH = `${process.env.PATH}${path.delimiter}${pathToAdd}`
Copy the code

DLL analysis tool

• 1. Dependency Walker

You can view all information about DLL link libraries, as well as DLL dependency tools, but unfortunately does not support WIN10. If you are not a WIN10 user, then you only need this tool, the following tool can be skipped.

• 2. Process Monitor

You can view various operations, such as I/O and registry access, during process execution. Libary is used to monitor the I/O operation of the node/electron process. Cause 3: Dynamic Linking Error: Win32

• 3. dumpbin

Dumpbin.exe is a Microsoft COFF binary converter that displays information about common Object File Format (COFF) binaries. You can use dumpbin to examine COFF object files, standard COFF object libraries, executables, and dynamically linked libraries, among others. Start with Visual Studio 20XX – Visual Studio Tools – VS20XX x86 Native Command Prompt from the Start menu.

Dumpbin /headers [DLL path] // returns DLL headers, indicating 32 bit Word Machine/64 bit Word Machine/ exports [DLL path] // returns DLL exports. The name list is the name of the exported functionCopy the code

Flash crash issue

During actual Node-ffi debugging, it is very easy to have memory errors flash, and even breakpoints cause crashes. This is often caused by illegal memory access, and you can see the error message in the Windows log, but trust me, it doesn’t help. C memory error is not a simple matter.

The appendix

Automatic conversion tool

Tjfontaine provides a node-ffi-generate function, which can automatically generate a node-ffi function declaration based on the header file. Note that this requires Linux environment, simple use of KOA package layer change to online mode ffi-online, can be put into VPS run.

WINAPI

The wheel

Winapi has a large number of custom variable types, waitingsong’s wheel Node-Win32-API has a complete translation of the full set of windef.h types, and this project uses TS to specify FFI return Interface, it is worth learning.

Attention! The types inside are not necessarily correct, I believe that the author has not tested all variables completely, and I have also encountered the pit of type error in actual use.

GetLastError

In short, Node-ffi calls DLLS through WinAPI, which causes GetLastError to always return 0. The easiest way to get around this problem is to write your own C++ addon.

Reference Issue GetLastError() always 0 when using Win32 API Reference PR github.com/node-ffi/no…

PVOID returns null, the memory addressFFFFFFFFFlash crash

In WinAPI, success is usually determined by the presence of the returned pvoid pointer, but in Node-ffi, deref() on FFFFFFFF causes the program to flash. You must bypass the pointer type of the pointer for special evaluation

HDEVNOTIFY
WINAPI
RegisterDeviceNotificationA( _In_ HANDLE hRecipient, _In_ LPVOID NotificationFilter, _In_ DWORD Flags);

HDEVNOTIFY hDevNotify = RegisterDeviceNotificationA(hwnd, &notifyFilter, DEVICE_NOTIFY_WINDOW_HANDLE);
if(! hDevNotify) { DWORD le = GetLastError();printf("RegisterDeviceNotificationA() failed [Error: %x]\r\n", le);
	return 1;
}
Copy the code

const apiDef = SetupDiGetClassDevsW: [W.PVOID_REF, [W.PVOID, W.PCTSTR, W.HWND, W.DWORD]] // Note that the return type 'w.pvoid_ref' must be set to pointer. If type is not set, Node-ffi will not attempt 'deref()'
const hDEVINFOPTR = this.setupapi.SetupDiGetClassDevsW(null, typeBuffer, null,
    setupapiConst.DIGCF_PRESENT | setupapiConst.DIGCF_ALLCLASSES
)
const hDEVINFO = winapi.utils.getPtrValue(hDEVINFOPTR, W.PVOID) // getPtrValue returns null if the address is full 'FF'
if(! hDEVINFO) {throw new ErrorWithCode(ErrorType.DEVICE_LIST_ERROR, ErrorCode.GET_HDEVINFO_FAIL)
}
Copy the code