Serverless ¿

We’ve been hearing this word every day for the last year or two on various public accounts and at front-end conferences. As soon as I hear what serverless, what micro-service, I always feel like a tourist of alien science and technology. No matter how good alien things are, how can I use them without such infrastructure in our unit? I am a little cutto, do I have to build a bunch of serverless facilities out?

Since the earth does not have that, we can only go to the alien stealthily learn a wave of skills, is the so-called “division yi long skills”, maybe one day when the Earth recruit also need engineers with less development experience.

What is a serverless?

Obscure alien language I’m not here to do more here, I believe you for what BaaS, FaaS has seen this word more than dozens of times, in the earth the person’s words is: have a serverless, you don’t have to go to care about what the server when it is applied in writing, the back-end operations, we only need to focus on writing business logic code. Let’s recap some of the most well-worn advantages

  1. Rapid application development
  2. Elastic expansion of service
  3. Charge by use

Hearing this, some people may think that serverless is so powerful, does the back-end and operation staff have to be laid off? This question is a matter of opinion. I can express my personal opinion: Under the premise of using serverless architecture on a large scale, serverless can liberate the back-end from the business and divide the responsibilities of different engineers more clearly for companies with serverless infrastructure. For companies that rely on others’ serverless services, JS full stack engineers are already competent for all business development responsibilities.

Serverless Select a service provider

Before the real experience, let’s compare the existing Serverless service providers

  1. Aws Lambda – Amazon Lambda was first launched in 2014. It is the most famous serverless computing service provider and once became the synonym of Serverless. Netflix (IQiyi) is one of the service’s better-known customers.
    • Free amount: per month1 millionSecondary request and400000 gb - secondsCalculation time of
    • Charges:$0.00001667 / GB - seconds

  1. Azure Functions – Microsoft launched their Serverless solution, Azure Functions, in 2016.

    • Free amount: per month1 millionSecondary request and400000 gb - secondsCalculation time of
    • Charges:$0.000016 / GB - seconds

  2. Google Cloud Functions – Google Cloud Functions was launched in 2017. It lagged behind Amazon and Microsoft in the early days, but has fixed a number of problems in recent years and is catching up.

    • Free amount: per month2 millionSecondary request and400000 gb - secondsCalculation time of
    • Charges:$0.0000004 / GB - seconds(Additional levymemorywithCPUThe cost of)
  1. IBM Cloud Functions – IBM’s latest solution based on the open source Serverless project Apache OpenWhisk.
    • Free amount: per month1 millionSecondary request and400000 gb - secondsCalculation time of
    • Charges:$0.000017 / GB - seconds
  1. Ali Cloud function computing – Ali Cloud first appeared in 2017 serverless project.
    • Free amount: per month1 millionSecondary request and400000 gb - secondsCalculation time of
    • Charges:$0.00001617 / GB - seconds(Based on current exchange rate 1:6.87)

From a price point of view, the pricier one is Google, whose extra charge for memory and CPU will significantly increase the cost of using its services, despite offering 2 million free requests. The prices of the other four are relatively close, among which Microsoft is the lowest, IBM is the highest, And Amazon and Ali Cloud are in the middle.

From a practical point of view, Amazon’s and Microsoft’s services are still well-equipped (multiple triggers, multiple languages supported…). And rich community support dominated most of the reviews, while Google continued to keep pace with up-and-comers IBM and AliYun. Of the five, Lambda is arguably the best choice to experience Serverless.

Serverless Application framework

We chose the Serverless framework to quickly create and deploy Lambda services.

Serverless here refers to a CLI tool that has over 30,000 stars on GitHub. Through Serverless CLI, we can quickly generate Lambda service templates, standardize and engineer the development of services and deploy multiple sets of environments and nodes with one click, greatly reducing the time of service development to the front line.

If you’re more of a believer in pure AWS infrastructure and want to follow the original AWS Lambda development documentation, that’s great too. It’s just that maybe my service goes live today and yours will have to wait until the day after tomorrow.











Build the service on Lambda

The target

In the next two hours, we will deploy a common set of user services on Lambda, consisting of the following four interfaces

  1. /api/user/signup– Create a new user and enter it into the database
  2. /api/user/login– Login and return a JSON Web Token to give the user access to the private interface
  3. /api/public– Public interface, accessible to users who do not need to log in
  4. /api/private– Private interface. Only login users can access it

Prepare the material

  1. A smooth global Internet
  2. An available AWS account
  3. According to theServerless documentComplete the first two steps
    • npm install -g serverless
    • Set the AWS Credentials

Create a project

The language we chose was JS, and the database was DynamoDb. Aws – Node-rest-API-with-Dynamodb is a template you can quickly find in serverless’s sample library

Copy the template locally as a start project

serverless install -u https://github.com/serverless/examples/tree/master/aws-node-rest-api-with-dynamodb
Copy the code

This project includes a Todo list CRUD operation service. The core files are:

  1. Serverless.yml defines the Lambda functions provided by the service, the triggers that trigger the function, and other AWS resources needed to run the function.

  2. Package. json defines the other libraries on which the service depends.

  3. The todos directory contains all the function files. We can see that the functions are fairly straightforward, and that each file has a structure similar to the following:

    const AWS = require('aws-sdk'); // Introduce the AWS SDK
const dynamoDb = new AWS.DynamoDB.DocumentClient(); // Create a dynamoDb instance

// exports this function through module.exports
module.exports.create = (event, context, callback) = > {
    const data = JSON.parse(event.body); // Parse the event to get the request data
    /*
      业务逻辑
    */
    callback(); // Use callback to return the response data
}

Copy the code

Once we have deployed the startup project to the cloud by running NPM Install and Serverless Deploy, we can use the Api address (for example: Xxxxxx.execute-api.us-east-1.amazonaws.com/dev/todos) to run and access these functions.

Create and define functions

It is not difficult to create similar function files based on the existing functions of the project. We create the following 4 files in the project:

  • user/signup.js
  • user/login.js
  • user/public.js
  • user/private.js

But it is not enough to just create the function files; we need to add definitions for these functions simultaneously in serverless.yml. Using signup as an example, add the following to functions:

signup:
    handler: user/signup.signup # define the path to the function file
    events:
      - http: # define the function trigger type as HTTP (AWS API Gateway)
          path: api/user/signup The request path is defined
          method: post The request method type is defined
          cors: true # Enable cross-domain
Copy the code

So we have fully defined four functions. Let’s look at the concrete implementation of these four functions.





The Public function

GET

Returns a message that can be accessed without logging in

1. Return a message

The public function is the easiest of the four because it is completely public and we do not need to do any validation on it. Simply return a message as follows:

// user/public.js
module.exports.public = (event, context, callback) = > {
  const response = {
    statusCode: 200.body: JSON.stringify({
      message: 'Anyone can read this message. '})}return callback(null, response);
};
Copy the code

Note: The first parameter of callback is passed in error, and the second parameter is passed in response data.

2. Deploy and access the service

  • Run the following command to deploypublicfunction 
    Deploy a single function
serverless deploy function -f public
Copy the code

    or

    Deploy all functions
serverless deploy
Copy the code
  • To send the request, enter the API address directly in your browser or use the cURL tool to run the following commandserverless deployAfter log or inAWS API Gateway console – StageFind the) 
    curl -X GET https://xxxxxx.execute-api.us-west-2.amazonaws.com/dev/api/public
Copy the code
  • Return the data 
    {
   "message": "Anyone can read this message."
}
Copy the code





Sign-up function

POST

Create a new user and log into the database, returning success or failure information

1. Define resources

Signup requires DynamoDB to run, so the first step is to make the following changes to resources in the serverless.yml file

# serverless.yml
resources:
  Resources:
    UserDynamoDbTable:
      Type: 'AWS::DynamoDB::Table' The resource type is DynamoDB
      DeletionPolicy: Retain Keep this table when deleting the CloudFormation Stack (Serverless remove)
      Properties:
        AttributeDefinitions: Define the attributes of the table
          -
            AttributeName: username # attribute name
            AttributeType: S The property type is string
        KeySchema: The primary key of the table
          -
            AttributeName: username The property name for the # key
            KeyType: HASH The key type is hash
        ProvisionedThroughput: Preset throughput of the table
          ReadCapacityUnits: 1 # Read 1 unit
          WriteCapacityUnits: 1 Write capacity is 1 unit
        The table name is DYNAMODB_TABLE as defined in the environment variable
        TableName: ${self:provider.environment.DYNAMODB_TABLE}
Copy the code

The resources column contains the AWS CloudFormation template written using YAML syntax.

For more details about the definition of DynamoDB table in CloudFormation, see the link.

2. Obtain request data

Signup is an interface whose method is POST, so you need to get the request data from the body that triggered the event.

// user/signup.js
module.exports.signup = (event, context, callback) = > {
  // Get the request data and parse the JSON string
  const data = JSON.parse(event.body);
  const { username, password } = data;
  / *... Verify username and passowrd */
}
Copy the code

3. Enter the user name to DynamoDB

After retrieving the requested data, we need to construct the new user’s data and enter it into DynamoDB

// user/signup.js
// Import the nodeJS encryption library
const crypto = require('crypto');
// Create a dynamoDB instance
const AWS = require('aws-sdk'); 
const dynamoDb = new AWS.DynamoDB.DocumentClient();

module.exports.signup = (event, context, callback) = > {
  / /... Get and verify username and password

  // Generate data for the new user
  
  // Generate a salt to ensure the uniqueness of the hashed password
  const salt = crypto.randomBytes(16).toString('hex');
  // Encrypt with sha512 hash function to generate a hash password that can only be authenticated one-way
  const hashedPassword = crypto.pbkdf2Sync(password, salt, 10000.512.'sha512').toString('hex');
  const timestamp = new Date().getTime(); // Generate the current timestamp
  const params = {
    TableName: process.env.DYNAMODB_TABLE, // Get the DynamoDB table name from the environment variable
    Item: {
      username, / / user name
      salt, // Save the salt for one-way verification password at login
      password: hashedPassword, // hash the password
      createdAt: timestamp, // Generate time
      updatedAt: timestamp // Update time}}// Input to dynamoDb
  dynamoDb.put(params, (error) => {
    // Return a failure message
    if (error) {
      // Log error information, which can be viewed in the AWS CloudWatch service
      console.error(error);
      callback(null, {
        statusCode: 500.body: JSON.stringify({
          message: 'Failed to create user! '})}); }else {
      // Returns a success message
      callback(null, {
        statusCode: 200.body: JSON.stringify({
          message: 'User created successfully! '})}); }}Copy the code

See the link for more detailed DynamoDB CRUD operation documentation in Nodejs

4. Deploy and access the service

  • Run the following command to deploysignupfunction 
    Deploy a single function
serverless deploy function -f signup
Copy the code

    or

    Deploy all functions
serverless deploy
Copy the code
  • withcURLThe tool executes the following command to send the request (replace it with your API address, which is available at runserverless deployAfter log or inAWS API Gateway console – StageFind the) 
    curl -X POST https://xxxxxx.execute-api.us-west-2.amazonaws.com/dev/api/user/signup --data '{ "username": "new_user", "password": "12345678" }'
Copy the code
  • Return the data 
    {
  "message": "success"
}
Copy the code





The Login function

GET

Validates the username and password and returns a JSON Web Token to allow the logged-in user to access the private interface

1. Set the JSON Web Token

After the user invokes the Login interface and is authenticated, we need to return a JSON Web Token for the user to invoke the service that requires permission. To set the JSON Web Token, perform the following steps:

  • npm install jsonwebtoken --saveInstall the JsonWebToken library and add it to the project dependencies
  • Add one to the projectsecret.jsonFile to store the key, here we use symmetric encryption to define a private key. 
    // secret.json
{
    "secret": "Private key"
}
Copy the code
  • Define the private key to an environment variable for the function to access. inserverless.ymltheproviderThe following changes are made 
    # serverless.yml
provider:
  environment:
    # Use the serverless variable syntax to assign the key in the file to the environment variable PRIVATE_KEY
    PRIVATE_KEY: ${file(./secret.json):secret}
Copy the code

2. Obtain request data

Login is an interface with a method of GET, so you need to GET the request data from the queryStringParameters that trigger the event.

// user/login.js
module.exports.login = (event, context, callback) = > {
  // Get the request data
  const { username, password } = event.queryStringParameters;
  / *... Verify username and passowrd */
}

Copy the code

3. Verify the password and return the JSON Web Token

// user/login.js
const crypto = require('crypto');
const jwt = require('jsonwebtoken');
const AWS = require('aws-sdk');
const dynamoDb = new AWS.DynamoDB.DocumentClient();

module.exports.login = (event, context, callback) = > {
  / /... Get and verify username and password
  
  // Validates the password and returns the JSON Web Token
  
  // create a DynamoDB request with the primary key username
  const params = {
    TableName: process.env.DYNAMODB_TABLE,
    Key: {
      username
    }
  };
  // Get data from DynamoDB
  dynamoDb.get(params, (error, data) => {
    if (error) {
      // Log error information, which can be viewed in the AWS CloudWatch service
      console.error(error);
      // Return an error message
      callback(null, {
        statusCode: 500.body: JSON.stringify({
          message: 'Login failed! '})});return;
    }
    
    // Get the user data returned by DynamoDB from the callback argument
    const user = data.Item;
    if (
        // Verify that username exists
        user &&
        // Verify that the hashed passwords match
        user.password === crypto.pbkdf2Sync(password, user.salt, 10000.512.'sha512').toString('hex')) {// Return login success message
      const response = {
        statusCode: 200.body: JSON.stringify({
          username, / / returns the username
          // Returns JSON Web Token
          token: jwt.sign(
            {
              username // Embed username data
            },
            process.env.PRIVATE_KEY // Use the private key to sign the token)})}; callback(null, response);
    } else {
      // Return an error message
      callback(null, {
        statusCode: 401.body: JSON.stringify({
          message: 'Wrong username or password! '})}); }}); };Copy the code

4. Deploy and access the service

  • Run the following command to deployloginfunction 
    Deploy a single function
serverless deploy function -f login
Copy the code

    or

    Deploy all functions
serverless deploy
Copy the code
  • To send the request, enter the API address directly in your browser or use the cURL tool to run the following commandserverless deployAfter log or inAWS API Gateway console – StageFind the) 
    curl -X GET 'https://xxxxxx.execute-api.us-west-2.amazonaws.com/dev/api/user/login?username=new_user&password=12345678'
Copy the code
  • Return the data 
    {
  "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6Im5ld191c2VyIiwiaWF0IjoxNTYxODI1NTgyfQ.Iv0ulooGayulxf_MkkpBO1xEw1g ilThT62ysuz-rQE0"."username": "new_user"
}
Copy the code





Auth function

Verify that the JSON Web Token contained in the request is valid

Before writing the private function, we need to provide another function, auth, to verify that the JSON Web Token in the user-submitted request matches the one we issued.

1. Create functions

  • Add in the projectuser/auth.jsfile
  • inserverless.ymlthefunctionsAdd the following: 
    auth:
    handler: user/auth.auth
# auth is a function that is only called within the service, so there are no triggers
Copy the code

2. Generate a response containing the IAM permission policy

In order for the AWS API Gateway trigger to correctly identify whether or not a function has permission to execute, we must return a response data with IAM (AWS Services and Permission Management System) permission policy information in the Auth function so that the function with permission can be successfully triggered by the AWS API Gateway. Define a method like this in user/auth.js:

// user/auth.js
const generatePolicy = (principalId, effect, resource) = > {
  const authResponse = {};
  authResponse.principalId = principalId; // Used to mark user identity information
  if (effect && resource) {
    const policyDocument = {};
    policyDocument.Version = '2012-10-17'; // Define version information
    policyDocument.Statement = [];
    const statementOne = {};
    statementOne.Action = 'execute-api:Invoke'; // Define the operation type, in this case the API call operation
    statementOne.Effect = effect; // The available values are ALLOW or DENY, which specify whether the result of the policy is allowed or denied
    statementOne.Resource = resource; // Pass ARN (AWS resource name) to specify the resources required for the operation
    policyDocument.Statement[0] = statementOne;
    authResponse.policyDocument = policyDocument; // Add the defined policy to the response data
  }
  return authResponse;
};

Copy the code

For more detailed configuration documents about the IAM policy, see the link

3. Parse and verify the JSON Web Token

Our default request when parsing JSON Web tokens follows the Bearer Token format in OAuth2.0.

// user/auth.js
const jwt = require('jsonwebtoken');

/ *... Define the generatePolicy method */

module.exports.auth = (event, context, callback) = > {
  // Get the Authorization in the request header
  const { authorizationToken } = event;
  if (authorizationToken) {
    / / parsing Authorization
    const split = event.authorizationToken.split(' ');
    if (split[0= = ='Bearer' && split.length === 2) {
      try {
        const token = split[1];
        // Verify the JSON Web Token with the private key
        const decoded = jwt.verify(token, process.env.PRIVATE_KEY);
        // Use generatePolicy to generate response data containing the IAM permission policy that allows API calls
        const response = generatePolicy(decoded.username, 'Allow', event.methodArn);
        return callback(null, response);
      } catch (error) {
        // JSON Web Token verification fails, and an error is returned
        return callback('Unauthorized'); }}else {
      // The Authorization format verification fails and an error is returned
      return callback('Unauthorized'); }}else {
      // Request header does not contain authorization, return error
      return callback('Unauthorized'); }};Copy the code





Private function

GET

Returns a message that requires login to access

The implementation of the private function is very similar to the previous public function, except that we need to add the auth we just defined as the permission verification function to the HTTP (AWS API Gateway) trigger of the function.

1. Set the authorizer

Make the following changes to the previously defined private function in serverless.yml:

# serverless.yml
functions: 
  private:
    handler: user/private.private
    events:
      - http:
          path: api/private
          method: get
          authorizer: auth # Set authorizer to auth function
          cors: true
Copy the code

2. Return the message

// user/private.js
module.exports.private = (event, context, callback) = > {
  // Get the requested user information from the trigger event
  const username = event.requestContext.authorizer.principalId;
  // Returns a message
  const response = {
    statusCode: 200.body: JSON.stringify({
      message: Hi, `${username}! Only logged in users can read this message. `})}return callback(null, response);
};
Copy the code

3. Deploy and access the service

  • Run the following command to deployprivatefunction 
    Deploy a single function
serverless deploy function -f private
Copy the code

    or

    Deploy all functions
serverless deploy
Copy the code
  • Using the cURL tool, run the following command to send the requestserverless deployAfter log or inAWS API Gateway console – StageFind the) 
    curl -X GET -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6Im5ld191c2VyIiwiaWF0IjoxNTYxODI1NTgyfQ.Iv0ulooGayulxf_MkkpBO1xEw1gi lThT62ysuz-rQE0" https://xxxxxx.execute-api.us-west-2.amazonaws.com/dev/api/private
Copy the code
  • Return the data 
    {
  "message": "Hello new_user! Only logged in users can read this message."
}
Copy the code





summary

What was used?

  1. serverless cli
  2. AWS Lambda
  3. AWS API Gateway
  4. AWS CloudWatch
  5. DynamoDB
  6. OAuth2.0

What did you do?

  1. Developed and deployed a set of API services
  2. The service provides user registration and login interfaces
  3. The service provides a fully open interface and a login – only accessible interface

Complete the project

  • The full engineering of the project is available at github.com/yuanfux/aws… In the view
  • Run the following command to quickly deploy the project and access the services
    1. serverless install -u https://github.com/yuanfux/aws-lambda-user
    2. cd aws-lambda-user
    3. npm install
    4. serverless deploy











thinking

We see AWS Lambda’s utility, convenience, and low cost, but can we see some of the problems behind it?

Practical ¿

  • Code maintenance

    The functional development pattern ensures that the reuse and sharing of code is a problem, and that the amount of code expands with the number of services. Serverless causes the number of functions to increase linearly with the amount of code, as shown in the figure below

When the number of services reaches a certain level, the additional manpower and expense required to maintain an infinitely expanding codebase is also significant.

  • Cold start

    Cold start is also a common topic. In simple terms, when your function has not been run for a while, the system reclaims the container resources used to run your function. The downside of this is that the next time you call this function, you need to reconfigure a container to run your function, and the result is that the call is delayed. Let’s look at the relationship between function call interval and cold start probability:

So what is the exact delay time? The latency depends on many factors, such as the programming language you choose, the memory configuration of the function, the file size of the function, and so on. But a common suggestion is that Lambda is not suitable for services that are extremely latency sensitive (< 50ms).

Convenient ¿

  • Local development

    Running Lambda functions relies on many external libraries and applications (AWS – SDK, API Gateway, DynamoDB…). Therefore, it is very difficult to run such functions in a completely native environment. If we had to deploy and rely on the run log output from AWS CloudWatch to debug and develop Lambda functions every time we changed the function, it would be extremely inefficient. Fortunately, serverless CLI provides plugins to support basic local development (serverless-offline, serverless-dynamodb-local…). . But Lambda users who don’t use the Serverless CLI may have to peruse the AWS-SAM documentation and go through a much longer configuration process.

  • The migration

    As you may have discovered by developing AWS Lambda, the code we write is very relevant to AWS, the cloud service provider. Despite the Serverless CLI, a common Serverless application framework that helps smooth out code differences between vendors, moving from one vendor to another is still a lot of manual labor, and even involves a lot of code refactoring. Using serverless is like smoking. You may enjoy the beauty of smoking at the beginning and feel that it doesn’t matter to smoke a few cigarettes. You can certainly quit when you want to. But as you get deeper into it, you’ll find that quitting is a pretty painful process.

Low price ¿

  • Valuation methods

    The minimum AWS Lambda charge is 100ms, which means your function will be billed as 100ms even if it only executes 1ms. This billing can even result in high-memory functions being cheaper than low-memory functions! Let’s take a look at AWS Lambda’s billing table:

    To take a more extreme example, let’s say we set up a function with 448MB of memory and 101ms of running time, then we pay 0.000000729 x 2 = $0.000001458 for each execution. If we increase the memory of this function to 512MB and reduce its running time by 100ms or less, we only pay $0.000000834 per execution. With just one setup, we reduced the cost by a full (1458-834) / 1458 ≈ 42.8%!

    Finding the most cost-effective memory setup means extra work, and it’s hard to imagine AWS failing to provide a reasonable solution to this problem for its customers.

  • Bound to consumption

    Almost all the peripheral services (API Gateway, CloudWatch, DynamoDB…) when using AWS Lambda There is an extra charge. One of the most obvious features is bundled consumption, and you might be hard-pressed to imagine CloudWatch being a mandatory service when using Lambda; The API Gateway can hardly escape a toll booth when setting up HTTP services, with a price tag of $3.5/million requests that is much higher than using Lambda.

  • Growth vampires

    Ming just migrated his $5 per month personal blog set up by a VPS vendor to AWS Lambda. He found that none of the services exceeded the AWS free line, and Lambda saved Ming $5 per month. For small traffic and small memory services like Xiaoming, Lambda will indeed save a considerable cost; But Lambda’s per-call charges can add up to a lot of money for services that use a lot of traffic and a lot of memory. Using a Lambda is like staying in a hotel, while renting a server is like renting. Hotels are well-equipped and convenient, and it may be cheaper to stay for a few days than rent, but I can’t afford to stay every day and every night.

delicious

While Lambda has these trade-offs, the efficiencies it brings to development are unprecedented, and the cost savings it brings to service development and operations are visible. If you don’t know what the front-end of serverless is, we usually only use two words to describe such people: omnipotent! I often say that today learn serverless, tomorrow can open a company. Read this article today and if you can’t write serverless services on Lambda, I will eat this computer screen on the spot.