Based on Node.js lightweight cloud function function implementation

Introduction:

In the age of everything in the cloud, your application doesn’t even need a server. Cloud function functions are provided in various cloud services, so how to use the “omnipotent” Node.js implementation?

What is cloud function?

Cloud function is a new term born from cloud service. As the name implies, cloud function is the function executed in the cloud (i.e. the server side). Each cloud function is independent, simple and has a single purpose, and its execution environment is isolated from each other. When using cloud functions, developers only need to focus on the business code itself. The rest, such as environment variables and computing resources, are provided by cloud services.

Why are cloud functions needed?

Programmers say they don’t want to buy servers, so they have cloud services; The programmer said he didn’t even want to write server, so there was a cloud function.

Serverless architecture

Usually our application, there will be a background program, it is responsible for processing various requests and business logic, generally need to deal with network, database and other I/O. In a serverless architecture, everything is handled by the execution environment except the business code. The developer doesn’t need to know how the server runs or how the DATABASE API is called — everything is written outside in a “greenhouse”.

FaaS

Cloud functions are how the Serverless architecture is implemented. Our application will be composed of separate functions, each of which is a small – grained business logic unit. There are no servers, no server programs, Functions as a Service.

Three, how to achieve?

Since this implementation is applied in a CLI tool, the function is declared in the developer’s project file, so the general process is as follows:

1. Function declaration and storage

The statement

Our goal is to have cloud functions declared just like normal JS functions:

module.exports = async function (ctx) {
    return 'hahha'}};Copy the code

Since cloud functions are usually executed with interface calls, it should be possible to declare HTTP methods:

module.exports = {
  method: 'POST'.handler: async function (ctx) {
    return 'hahha'}};Copy the code

storage

The handler field is an object of type Function. The handler field is a Function object. We can call its toString method to get the body of the function as a string:

const f = require('./func.js');
const method = f.method;
const body = f.handler.toString();
// async function (ctx) {
// return 'hahha'
// }
Copy the code

With the string function body, storage is very simple, directly stored in the database string field can be.

2. Function execution

url

If used for front-end calls, each cloud function needs to have a corresponding URL. If the file name of the above declaration file is the unique name of the cloud function, the URL can be simply designed as:

/f/:funcname
Copy the code

Construct independent scope (emphasis)

In the JS world, there are several ways to execute a function body as a string:

1. evalfunction
2. new Function
3. vmThe module

So which one to choose? Let’s review the characteristics of cloud functions: they are independent, independent and run in the cloud. The key is to execute each cloud function in a separate scope with no access to the execution environment, so nodeJS vm modules are the best choice. You can refer to the official documentation for the use of this module. So far, the execution of cloud functions can be divided into three steps:

1. Get the function body from the database
2. structurecontext

// CTX is the context object for KOA
const sandbox = {
    ctx: {
      params: ctx.params,
      query: ctx.query,
      body: ctx.request.body,
      userid: ctx.userid,
    },
    promise: null.console: console
  }
  vm.createContext(sandbox);
Copy the code

3. Execute the function to get the result

const code = `func = ${funcBody}; promise = func(ctx); `;
vm.runInContext(code, sandbox);
const data = await sandbox.promise;
Copy the code

The VM2 module of the NPM community addresses some of the security flaws of the VM module and can also be used in the same way.

3, references,

In principle, cloud functions should be independent of each other, but for flexibility, we have decided to support cross-references between functions, where one cloud function can be called from another.

The statement

Simply add an array field with the name of the function:

module.exports = {
  method: 'POST'.use: ['func1'.'func2'].handler: async function (ctx) {
    return 'hahha'}};Copy the code

injection

It is also very simple, according to the dependency chain to find all functions, all mounted under CTX, depth first or breadth first can be good.

if (func.use) {
	const funcs = {};
	const fnames = func.use;
	for (let i = 0; i < fnames.length; i++) {
		const fname = fnames[i];
		await getUsedFuncs(ctx, fname, funcs);
	}

	const funcCode = ` {The ${Object.keys(funcs).map(fname => `${fname}:${funcs[fname]}`).join('\n')}} `;

	code = `ctx.methods=${funcCode};${code}`;
} else {
	code = `ctx.methods={};${code}`;
}

// Get all dependent functions
const getUsedFuncs = async (ctx, funcName, methods) => {
	const func = getFunc(funcName);
	methods[funcName] = func.body;
	if (func.use) {
		const uses = func.use.split(', ');
		for (let i = 0; i < uses.length; i++) {
			awaitgetUsedFuncs(ctx,uses[i], methods); }}}Copy the code

Depend on the loop

Since it is possible to rely on each other, it is inevitable that a→ B → C → A dependency cycle will occur, so it is necessary to detect the dependency cycle when the developer submits the cloud function. The idea of detection is also very simple. In the process of traversing the dependent chain, each individual chain is recorded. If the function currently traversed is found to have appeared in the chain, the cycle will occur.

const funcMap = {};
flist.forEach((f) = > {
	funcMap[f.name] = f;
});

const chain = [];
flist.forEach((f) = > {
	getUseChain(f, chain);
});

function getUseChain(f, chain) {
	if (chain.includes(f.name)) {
		throw new Error(The function has a loop dependency:${[...chain, f.name].join('- >')}`);
	} else {
		f.use.forEach((fname) = >{ getUseChain(funcMap[fname], [...chain, f.name]); }); }}Copy the code

4, performance,

In the above scenario, the following steps are required each time the cloud function is executed:

1. Get function body
2. Compile the code
3. Tectonic scope and independent environment
4. perform

Step 3: Because the parameters of each execution are different and the same function may be executed concurrently with different requests, the scope CTX cannot be reused. Step 4 is required, which leaves the optimizable points 1 and 2.

Code cache

The VM module provides an interface for code compilation and execution to be handled separately, so each time a function body string is obtained, it is compiled into a Script object:

/ /... get code
const script = new vm.Script(code);
Copy the code

We can pass in a compiled Script object directly to execute:

/ /... get sandbox
vm.createContext(sandbox);
script.runInContext(sandbox);
const data = await sandbox.promise;
Copy the code

Function body cache

Simple cache, does not need very complex update mechanism, set a time threshold, after the new function body and compile Script object, and then cache it:

const cacheFuncs = {};
/ /... get script
cacheFuncs[funcName] = {
	updateTime: Date.now(),
	script,
};

// cache time: 60 sec
const cacheFunc = cacheFuncs[cacheKey];

if (cacheFunc && (Date.now() - cacheFunc.updateTime) <= 60000) {
	const sandbox = { / *... * / }
	vm.createContext(sandbox);
	cacheFunc.script.runInContext(sandbox);
	const data = await saandbox.promise;
	return data;
} else {
	// renew cache
}
Copy the code

Iv. Reference materials

Related articles

What is the Serverless architecture?

The industry of serverless

Tencent Cloud – No service Cloud function ali Cloud – Function computing AWS – Lambda Azure – Azure Functions