Log alarm using Serverless

A recent attempt was made to implement the application of the page JS error alarm function through Serverless. This article mainly introduces the concrete implementation process, and some problems encountered.

The requirement of the alarm function is also very simple, namely to read the ARMS error log at a regular time (e.g., every 1 minute), and if there is an error log, to alarm by sending the error details through the pin message.

Before that, I implemented this feature with timed tasks. In terms of cost, this solution requires the application of a separate server resources; Moreover, scheduled tasks are executed only at the appropriate time, which means that the server is idle for a long time, resulting in a waste of resources. With Serverless, there is no need to apply for a server and functions only need to be executed when needed, resulting in significant cost savings.

In general, I think the advantages of function calculation are:

For developers, only the implementation of business logic needs to be concerned, not the environment in which the code runs, hardware resources, and operations
Cost savings

The front-end log alarm is realized through Serverless. The cloud service is Ali Cloud function computing, and other tools are also relied on:

Fun, a command-line tool for function calculation, is used to debug and deploy functions locally
Interactive tool for function calculation, fCLI, for local testing
The JS SDK aliyun-SDK-js is used to read SLS logs. The logs of ARMS are stored in SLS
The programming language uses Node.js

Install and configure Fun

Fun will need to be installed for first use

$ npm install @alicloud/fun -g
Copy the code

After installing Aliyun Account ID Aliyun Access Key ID Aliyun Secret Access Key and default region, run fun Config to configure the Account information. One thing to note here is that if you want to use SLS to log functions, you need the SLS and the function service to be in the same region. More on this later.

$ fun config
? Aliyun Account ID ******
? Aliyun Access Key ID ******
? Aliyun Secret Access Key ******
? Default region name cn-shanghai
? The timeout in seconds for each SDK client invoking 60
? The maximum number of retries for each SDK client 6
Copy the code

Aliyun Account ID Aliyun Access Key ID Aliyun Secret Access Key can be searched and set in the console of Aliyun Cloud.

Aliyun Account ID

! [Aliyun Account ID]

Aliyun Access Key ID Aliyun Secret Access Key

Function initialization

Create a Demo of Node.js with Fun and build on it.

$ fun init -n alarm helloworld-nodejs8
Start rendering template...
+ /Users/jh/inbox/func/code/alarm
+ /Users/jh/inbox/func/code/alarm/index.js
+ /Users/jh/inbox/func/code/alarm/template.yml
finish rendering template.
Copy the code

After successful execution, two files index.js and template.yml are created respectively.

Template. yml is the function specification document, which defines the resources needed by the function, the events that trigger the function, and so on.

template.yml

Next take a quick look at the generated default template.yml configuration file.

ROSTemplateFormatVersion: '2015-09-01'
Transform: 'Aliyun::Serverless-2018-04-03'
Resources:
  alarm:
    Type: 'Aliyun::Serverless::Service'
    Properties:
      Description: 'helloworld'
    alarm:
      Type: 'Aliyun::Serverless::Function'
      Properties:
        Handler: index.handler
        Runtime: nodejs8
        CodeUri: 'index.js'
Copy the code

We first define the version of the specification document, ROSTemplateFormatVersion and Transform, neither of which need to be changed.

Resources defined inside a function called alarm Service (Type: Aliyun: : Serverless: : Service said the properties serve function), and the Service defines a function called alarm (Type: ‘Aliyun: : Serverless: : Function indicates that the properties of Function).

A function service can contain multiple functions, which is equivalent to a group of functions. Function logging, which we will refer to later, is configured to function services. All functions in the function service use the same log.

You can modify the function service name and function name as required. Now change the function service name to Yunzhi and keep the function name as alarm.

ROSTemplateFormatVersion: '2015-09-01'
Transform: 'Aliyun::Serverless-2018-04-03'
Resources:
  yunzhi: The name of the function service
    Type: 'Aliyun::Serverless::Service' # indicates that Yunzhi is a function service
    Properties:
      Description: 'helloworld' # Description of function services
    alarm: The name of the function
      Type: 'Aliyun::Serverless::Function' # indicates that alarm is a function
      Properties:
        Handler: index.handler Function call entry
        Runtime: nodejs8 The environment in which the function is run
        CodeUri: 'index.js' # directory of code
Copy the code

Properties in the alarm function defines the call entry, run environment, and so on, as shown in the comments above.

See Serverless Application Model for template.yml configuration.

index.js

The index.js file is the entry point to the function. Index. handler indicates that the function calls the handler function in the index.[extension] file.

module.exports.handler = function(event, context, callback) { 
  console.log('hello world');
  callback(null.'hello world'); 
};
Copy the code

After initialization, the code is just a few lines above, very simple. The main point is to understand the above parameters.

eventArguments passed in when a function is called
contextSome information about the function at run time
callbackThe callback after the function is executed
- 1. It has to be calledcallbackFunction, is considered finished. If not called, the function runs until timeout
- 1. callbackAfter the call, the function ends
- 1. callbackThe first argument to iserrorObject, which is consistent with the idea of JS callback programming

For details on events and contexts, see the Nodejs function entry.

The main logic to implement the alarm function is written in index.js. Specific implementation, I will not go into details, the following pseudocode to describe:

alarm/alarm.js

// alarm/alarm.js 
// Implement the alarm function
module.exports = function() {
	return new Promise((resolve, reject) = > {
		// Query SLS logs
		// - Resolve if there is no error log
		// - Send a pin message if there is an error log
		Reject if the pin message fails to be sent
		Resolve if the pin message is sent successfullyresolve(); })}Copy the code

alarm/index.js

// alarm/index.js 
// Call the alarm function
const alarm = require('./alarm');

module.exports.handler = function(event, context, callback) { 
  alarm()
  	.then((a)= > {
			callback(null.'success');   
  	})
  	.catch(error= >{ callback(error); })};Copy the code

CodeUri

Const alarm = require(‘./alarm’); const alarm = require(‘./alarm’); , you need to change the default codeUri to the current code directory./. Otherwise, the default codeUri only defines index.js, and only index.js will be deployed.

ROSTemplateFormatVersion: '2015-09-01'
Transform: 'Aliyun::Serverless-2018-04-03'
Resources:
  yunzhi: The name of the function service
    Type: 'Aliyun::Serverless::Service' # indicates that Yunzhi is a function service
    Properties:
      Description: 'helloworld' # Description of function services
    alarm: The name of the function
      Type: 'Aliyun::Serverless::Function' # indicates that alarm is a function
      Properties:
        Handler: index.handler Function call entry
        Runtime: nodejs8 The environment in which the function is run
        CodeUri: '/' # directory of code
Copy the code

If the CodeUri is not modified, an error similar to the following is reported

$ fun local invoke alarm
FC Invoke End RequestId: 16e3099e-6a40-43cb-99a0-f0c75f3422c6
{
  "errorMessage": "Cannot find module './alarm'"."errorType": "Error"."stackTrace": [
    "Error: Cannot find module './alarm'"."at Module._resolveFilename (module.js:536:15)"."at Module._load (module.js:466:25)"."at Module.require (module.js:579:17)"."at require (internal/module.js:11:18)"."at (/code/index.js:9:15)"."at Module._compile (module.js:635:30)"."at Module._extensions.. js (module.js:646:10)"."at Module.load (module.js:554:32)"."at tryModuleLoad (module.js:497:12)"."at Module._load (module.js:489:3)"]}Copy the code

Fun Local Invoke Alarm is a local debug command, which is covered next.

Local debugging

Local debugging is definitely needed during development. Fun provides Fun Local support for local debugging.

Fun Local’s command format is Fun Local invoke [options] <[service/]function>, where options and service can be ignored. For example, the command to debug the alarm function above is Fun Local Invoke Alarm.

Note that local debugging requires docker to be installed first.

$ brew cask install docker
Copy the code

Start Docker after successful installation.

If Docker is not started, running Fun Local may cause the following error

$ fun local invoke alarm
Reading event data from stdin, which can be ended with Enter then Ctrl+D
(you can also pass it from file with -e)
connect ENOENT /var/run/docker.sock
Copy the code

The normal output is as follows

$ fun local invoke alarm
Reading event data from stdin, which can be ended with Enter then Ctrl+D
(you can also pass it from file with -e) skip pulling image aliyunfc/ Runtime-nodejs8:1.5.0... FC Invoke Start RequestId: 9360768c-5c52-4bf5-978b-774edfce9e40 load codefor handler:index.handler
FC Invoke End RequestId: 9360768c-5c52-4bf5-978b-774edfce9e40
success

RequestId: 9360768c-5c52-4bf5-978b-774edfce9e40          Billed Duration: 79 ms          Memory Size: 1998 MB    Max Memory Used: 54 MB
Copy the code

The first debugging will install a runtime image, which may take some time. The default Docker image download will be slow, you can use the domestic acceleration site to speed up the download.

When the message “Reading event data from STdin, which can be ended with Enter then Ctrl+D” is displayed, if you do not need to Enter it, press Ctrl+D to skip it.

Function deployment

After the completion of development, it is necessary to deploy the function to ali Cloud function calculation above. You can deploy using the fun deploy command.

After installing Fun, you have configured the aliyun account and region information using the fun config command. Fun Deploy will automatically deploy the function to the corresponding account and region.

In template.yml, the service name and function name of the function are also configured. If there is no corresponding service or function in the function evaluation, Fun Deploy is automatically created; If it already exists, it will be updated.

$ fun deploy
using region: cn-shanghai
using accountId: ***********4698
using accessKeyId: ***********UfpF
using timeout: 60

Waiting for service yunzhi to be deployed...
        Waiting for function alarm to be deployed...
                Waiting for packaging function alarm code...
                package function alarm code done
        function alarm deploy success
service yunzhi deploy success
Copy the code

After successful deployment, you can see the corresponding function services and functions in the function evaluation console. Triggers are not configured yet and can be manually executed by clicking the “Execute” button in the console.

The trigger

For functions applied to the production environment, it is definitely not executed manually as above, but by configuring triggers. A trigger is a specific event that a function invokes when it receives the event.

Alicloud’s function calculation supports HTTP triggers (functions are called upon receiving HTTP requests), timing triggers (functions are called on timing), OSS triggers, and so on. See the list of triggers.

For the alarm function, you need to use a timed trigger because you need to call the function at regular intervals.

Triggers are configured into functions and can be configured using the Event attribute of the function

ROSTemplateFormatVersion: '2015-09-01'
Transform: 'Aliyun::Serverless-2018-04-03'
Resources:
  yunzhi:
    Type: 'Aliyun::Serverless::Service'
    Properties:
      Description: 'helloworld'
    alarm: 
      Type: 'Aliyun::Serverless::Function'
      Properties:
        Handler: index.handler
        Runtime: nodejs8 
        CodeUri: '/' 
      Events: Configure the trigger for the alarm function
        TimeTrigger: The trigger name
          Type: Timer # indicates that the trigger is a timed trigger
          Properties: 
            CronExpression: "0/1 * * * *"  Run this command every 1 minute
            Enable: true # Whether to enable the timing trigger
Copy the code

The above configuration configures a timer trigger called TimeTrigger for Alarm that executes every minute, or calls a function every minute.

Once configured, execute Fun Deploy to publish functions and triggers to function calculations.

It should be noted here that ali Cloud function computing service currently supports triggers with a minimum interval of 1 minute. If less than 1 minute, the setting fails. For details about timing triggers, see the documentation for timing trigger functions.

Function log

For Serverless applications, we don’t really know which server our functions are running on, although we don’t care about operations. At this point, the log of the function is especially important. Without logs, it’s hard to know what’s going on, and there’s no way to start.

So next you need to configure logging for the function. Ali Cloud function calculation can use Ali cloud log service SLS to store logs. To store logs, enable the log service.

No log library exists

If you are using the logging service for the first time, there must be no log library. Logging resources can be defined by Resource in template.yml as well as function services.

As mentioned earlier, function logging is configured to the corresponding service, and the configuration is simple through the LogConfig property of the function service.

The complete template.yml is shown below

ROSTemplateFormatVersion: '2015-09-01'
Transform: 'Aliyun::Serverless-2018-04-03'
Resources:
  log-yunzhi: # Log project named log-yunzhi
    Type: 'Aliyun::Serverless::Log' # Indicates that the resource is the log service of Aliyun
    Properties:
      Description: 'yunzhi function service log project'
    log-yunzhi-store: # logStore for logs
      Type: 'Aliyun::Serverless::Log::Logstore'
      Properties:
        TTL: 10
        ShardCount: 1
    log-another-logstore: # another logstore for logs
      Type: 'Aliyun::Serverless::Log::Logstore'
      Properties:
        TTL: 10
        ShardCount: 1
  yunzhi:
    Type: 'Aliyun::Serverless::Service'
    Properties:
      Description: 'helloworld' 
      LogConfig: Configure function logs
        Project: 'log-yunzhi' # Store function log SLS project: log-yunzhi
        Logstore: 'log-yunzhi-store' # SLS logstore for storing function logs: log-yunzhi-store
    alarm:
      Type: 'Aliyun::Serverless::Function'
      Properties:
        Handler: index.handler 
        Runtime: nodejs8 
        CodeUri: '/' 
      Events: 
        TimeTrigger: 
          Type: Timer 
          Properties: 
            CronExpression: "0/1 * * * *"  
            Enable: true 

Copy the code

In the above configuration, a logging Project (Project) named log-yunzhi is defined and two log stores (logstores) are created in this Project: log-yunzhi-store and log-yunzhi-store. A Project can contain multiple LogStores.

Note: The name of the log item must be globally unique. That is, og-Yunzhi project name is globally unique in configuration.

After fun deploy is executed, log Project and log logStore are automatically created in the region corresponding to the function service, logStore is automatically indexed, and log stores are automatically configured for the function service.

The subsequent run logs of the function are stored in the corresponding LogStore.

$ fun deploy
using region: cn-shanghai
using accountId: ***********4698
using accessKeyId: ***********UfpF
using timeout: 60

Waiting for log service project log-yunzhi to be deployed...
        Waiting for log service logstore log-yunzhi-store to be deployed...
                retry 1 times
                Waiting for log service logstore log-yunzhi-store default index to be deployed...
                log service logstore log-yunzhi-store default index deploy success
        log serivce logstore log-yunzhi-store deploy success
        Waiting for log service logstore log-another-logstore to be deployed...
                Waiting for log service logstore log-another-logstore default index to be deployed...
                log service logstore log-another-logstore default index deploy success
        log serivce logstore log-another-logstore deploy success
log serivce project log-yunzhi deploy success

Waiting for service yunzhi to be deployed...
        Waiting for function alarm to be deployed...
                Waiting for packaging function alarm code...
                package function alarm code done
                Waiting for Timer trigger TimeTrigger to be deployed...
                function TimeTrigger deploy success
        function alarm deploy success
service yunzhi deploy success
Copy the code

If the log library already exists and log resources are defined, Fun Deploy updates the log library as configured in template.yml.

Log library exists

If the log library already exists, that is, the log Project and the log library Logstore have been created in the log service, you can directly add LogConfig to the function service without defining log resources.

Note that the log library needs to be in the same Region as the function service. Otherwise, the deployment fails.

Here is an example of configuring function logging to an existing Project and Logstore.

ROSTemplateFormatVersion: '2015-09-01'
Transform: 'Aliyun::Serverless-2018-04-03'
Resources:
  yunzhi:
    Type: 'Aliyun::Serverless::Service'
    Properties:
      Description: 'helloworld' 
      LogConfig: Configure function logs
        Project: 'log-yunzhi-exist' # Save function logs to existing Project: log-yunzhi-exist
        Logstore: 'logstore-exist' # store function logsto existing logStore: logstore-exist
    alarm:
      Type: 'Aliyun::Serverless::Function'
      Properties:
        Handler: index.handler 
        Runtime: nodejs8 
        CodeUri: '/' 
      Events: 
        TimeTrigger: 
          Type: Timer 
          Properties: 
            CronExpression: "0/1 * * * *"  
            Enable: true 

Copy the code

If the log library and the function service are not in the same locale, the function service will not find the log library and Fun Deploy will report an error. As shown below, Yunzhi-log-Qingdao is a log Project of Qingdao region created by me.

$ fun deploy
using region: cn-shanghai
using accountId: ***********4698
using accessKeyId: ***********UfpF
using timeout: 60

Waiting for service yunzhi to be deployed...
        retry 1 times
        retry 2 times
        retry 3 times
        retry 4 times
        retry 5 times
        retry 6 times
        retry 7 times
PUT /services/yunzhi failed with 400. requestid: 6af2afb8-cbd9-0d3e-bf16-fe623834b4ee, message: project 'yunzhi-log-qingdao' does not exist.
Copy the code

conclusion

Once the function logging is configured, the function is deployed to the function computation and is officially released.

Looking back, the process was relatively simple. But the process of going from scratch to deployment was fraught with problems. For example, many of the notes in the article are the result of constant trial and error.

The topic of Serverless has been very hot recently, and I am looking forward to the changes this technology will bring.

More github.com/nodejh/node…

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Install and configure Fun

Function initialization

template.yml

index.js

CodeUri

Local debugging

Function deployment

The trigger

Function log

No log library exists

Log library exists

Other problems

The child account AccessDenied

Code version management

Environmental problems

The economic costs

conclusion

Log alarm using Serverless

Install and configure Fun

Function initialization

template.yml

index.js

CodeUri

Local debugging

Function deployment

The trigger

Function log

No log library exists

Log library exists

Other problems

The child account AccessDenied

Code version management

Environmental problems

The economic costs

conclusion

Related Posts

How can I quickly remove unnecessary NPM packages from a project

Use vue-cookies to operate cookies and develop projects

Build a front-end team component system from 0 to 1 (Advanced Advanced Prerequisite)