Summary: At present, some evaluation agencies have put forward 40 criteria to evaluate the service capability of Serverless. These evaluation rules are not only a manifestation of the flourishing development of technology ecology, but also a reference for new entrants to evaluate the effectiveness of Serverless implementation.

Editor’s note: Two years ago, we were talking about what Serverless was and how it landed. At present, some evaluation agencies have given 40 criteria to evaluate Serverless’s service capability. These evaluation rules are not only a manifestation of the prosperous development of technology ecology, but also a reference for new entrants to evaluate the effectiveness of Serverless implementation.

For Forrester’s Function as a Service (FaaS) platform evaluation report, we selected nine of the most influential providers — Alibaba, Amazon, Google, Huawei, IBM, Microsoft, Nimbella, Oracle, and Tencent — and researched, analyzed, and scored them on 40 criteria. The report is designed to help application development and delivery (AD&D) professionals find the provider that best meets their needs by showing how each provider is performing in all aspects.

Forrester Wave™ : Function as a Service (FaaS) Platform First quarter 2021 Report

The FaaS platform helps developers quickly create cloud-native services

The abstraction of the FaaS platform allows developers to quickly create cloud-native microservices without having to worry about complex container or virtual machine cluster management and scaling. By leaving the management of the underlying infrastructure to the FaaS provider, developers can write microservices into simple little functions in a programming environment using familiar languages such as Java, C#, JavaScript, or Python. FaaS providers then automatically expand or shrink these microservices based on service requirements. Developers using the FaaS platform say that by eliminating infrastructure management and eliminating the complex operations associated with it with abstraction, they can quickly push new ideas into the deployment phase while determining infrastructure costs based on the actual resource requirements for performing microservices. When selecting a FaaS provider, the developer should analyze whether the provider meets the following criteria:

  • Support for function and container packaging. As developers deploy more and more types of workloads to the FaaS platform, the FaaS platform should allow developers to simply package a function as a ZIP or JAR file and deploy it, or to package custom code as an open Container standard (OCI) compliant container and deploy the corresponding framework. The FaaS platform should support both options to provide maximum flexibility for developers in developing and deploying Web, content, and event-driven workloads.
  • Provides robust security features. As developers expand their use of the FaaS platform, it is important to ensure secure access to data and application programming interfaces (apis) encapsulated in virtual private networks, or virtual private clouds (VPCS). In addition, when the function is scaled down, relevant personnel need to be able to quickly access these resources without waiting for the time-consuming “cold start”.
  • Support for third-party ecosystems and open standards. Unless you are willing to rely entirely on a public cloud provider, you should choose a FaaS provider with easier platform integration. Features you need to focus on include third-party observability, event binding, and messaging protocols.

To assess the

That’s not true, either. The Forrester Wave™ evaluation falls into the quadrants of Leaders, Strong Performers, competitors and Challengers. This is an assessment of the top vendors in the market and does not represent the market as a whole. You can check out our report on Serverless technologies for more information on this market. We hope that this evaluation report is just a starting point and recommend that customers use Excel based vendor comparison tools to view product evaluations and adjust standard weights (see Figures 1 and 2). Click the link at the beginning of the web version of the report on Forrester.com to download the tools.

Figure 1. Forrester Wave™ : Function as a Service (FaaS) Platform, Q1 2021

* Gray circles indicate vendors that did not participate in Forrester’s evaluation

Figure 2 Forrester Wave™ : Function as a Service (FaaS) Platform, Q1 2021

All ratings are scored on a scale of 0 (weak) to 5 (strong)

* Represents vendors that did not participate in the evaluation

Products from various manufacturers

In total, Forrester evaluated products from nine vendors: Alibaba, Amazon, Google, Huawei, IBM, Microsoft, Nimbella, Oracle, and Tencent (see Figure 3).

Figure 3 Vendors and products for this evaluation

Vendor profile

Through the analysis, we found that each manufacturer has the following advantages and disadvantages.

The leader

  • Amazon leads in terms of geographic reach, ecosystem breadth and execution stability. In our 2020 Forrester Wave™ report, we noted Lambda’s strong performance in terms of geographic availability, customer adoption, developer deployment function experience, and ecosystem support. These advantages also continue through 2021. In addition, features such as Amazon Elastic File System (EFS) support, the ability to run OCI-compliant containers, and billing granularity down to 1 millisecond keep Lambda on the road to innovation, making it available to more types of workloads. Strong support for serverless architecture security and special workloads keeps Lambda in the lead in 2021. Developers who subscribe to Amazon’s view that FaaS workloads should be short-lived will find Lambda provides them with a powerful FaaS platform. The platform is increasingly supporting third-party tools and container-centric deployment processes. The platform also offers unique services for refined functions such as state management, database operations, and apis.

Reference customers gave Amazon Web Services (Amazon Cloud Computing Services, or AWS) Lambda high marks for providing top-notch customer support, excellent performance, and integration with various AWS Services and third-party Services. The product has also been consistently appreciated by customers in maintaining close cooperation with experienced industry insiders. Reference customers saw room for improvement in observability and expected more simplified tools for developers. AWS Lambda is best suited for AD hoc workloads that run asynchronously, including integration with other AWS services, the Internet of Things, batch processing, new Web applications, and event-driven integration.

  • Alibaba uses containers and open standards to speed up Web and content workloads. In our 2020 report, we described Alibaba’s Function Compute as “a robust universal platform.” In 2021, we saw further enhancements in the platform’s support for containers and open source projects and standards, including OpenTracing, OpenTelemetry, Grafana, and Jaeger. Together with the platform’s investment in supporting mainstream Web frameworks, content-centric workloads, 1-millisecond billing granularity, and runtime, we put Alibaba function computing in the “leader” quadrant of the FaaS market this time. Alibaba function computation is particularly suited for workloads that require a large number of videos and leverages a container-centric model for function concurrency. For customers with such needs, the platform is a good choice.

Reference customers appreciated alibaba function computing’s ability to integrate events in China’s ISV ecosystem, and praised its efforts to strengthen customer support during holidays such as Christmas and New Year. On the technology side, customers were pleased with the ability to scale up to tens of thousands of instances in minutes. Reference customers wanted the platform to provide better programming language support for remote debugging and improved support for GPU computing. While the platform is currently best suited for companies deploying workloads in Hong Kong or Mainland China, its international reach is growing, with 10 geographies now supported and a total of 21 zones available.

  • Microsoft Azure Functions retains significant enterprise-level functionality for advanced initiatives. Consumer plans for Azure Functions support multiple programming languages and stateful Durable Functions, but developers need to upgrade to advanced dedicated plans if they want to use reserved instances, support for Docker containers, and powerful security features. The Azure App Service initiative. Persistence functions provide stateful functionality and support for binding to the Azure Event Hub, which helps developers build event-driven microservices. This year, Microsoft did well in the developer experience category, earning the highest score, by investing in improved new user orientation, more tools for developers, and support for development and debugging on local computers.

Reference customers are highly aware of Azure Functions’ ability to integrate with other Azure services such as Key Vault for confidential storage, specifically citing its powerful built-in observability. Customers were also generally satisfied with Azure Functions performance and quality of service. However, customers complain that they have to choose higher-cost premium service plans if they want to use important enterprise functions such as VPC services and reserved instances. Azure Functions are the most appropriate solution for customers who need other Azure services or a global FaaS platform. But if your team wants to build or integrate enterprise-level workloads, it needs to upgrade to the advanced version of the application service plan.

Strong performer

  • Tencent cloud Function (SCF) platform optimizes cloud-centric development. Tencent has built an excellent set of Web-based tools for function development and combined them with reliable runtimes that support multiple programming languages, complex workflows and longer running function instances. Developers can reserve function instances, and Tencent also provides professional support for artificial intelligence (AI) workloads and specific industry clouds. SCF can also serve as the extension mechanism of wechat ecosystem, effectively helping enterprises interact with customers through wechat extension functions. But for developers who want to develop and debug native functions, Tencent’s cloud-centric development philosophy doesn’t fit well with them.

Reference customers highly rated SCF for performance, customer support, and low cost of workload execution. Customers also expressed concerns about the stability of the platform and wanted to use more cloud services from Tencent and third parties with more built-in event triggers. Tencent SCF is a good choice for customers who need to deploy FaaS services in China, integrate and expand with wechat to attract more customers and embrace a cloud-centric development approach.

  • Nimbella fully supports hybrid clouds with a “FaaS everywhere” approach. Like IBM Cloud Functions, Nimbella is based on the open source Apache OpenWhisk project. Nimbella has a simple and quick developer entry process, and its runtime can be used under a pure FaaS model, deployed as an installable function platform to other public clouds, or deployed locally. Nimbella implements “FaaS everywhere” with powerful container support, multiple programming languages, and configurable maximum running time Settings for functions. Such a FaaS solution is ideal for stateful functions that need to run for a long time. Although the platform is very flexible, there are some issues that need to be addressed. Its managed solutions offer built-in isolation for tenants, while installable solutions support mixed deployment, with teams tailored to different organizational needs and security policies. As a result, teams that opt for a hybrid deployment may spend more time configuring the infrastructure for their production environment.

Reference customers spoke highly of Nimbella’s performance in customer support and helping to implement best practices. Customers also cited minimizing vendor lock-in risk as a reason for choosing Nimbella. The customer’s concerns included security and observability, and wanted to speed things up by adding code examples and digital content. Nimbella is a suitable solution for development teams that adopt a hybrid FaaS solution and want to provide their own networking and integration capabilities on top of a serverless architecture platform. Nimbella’s license model is also the most attractive option if you’d rather pay a monthly fee or opt for an enterprise agreement than a usage-based pricing model.

  • Google creates “all-encompassing” solutions with Cloud Functions and Cloud Run. As you can see from Google’s FaaS offering, multiple FaaS platforms are beginning to transform as public cloud providers use containers and the Kubernetes platform as key platform engines for THE FaaS runtime. Google’s Cloud Functions support common function packaging and deployment, their runtime scales well and cools slowly, and the Google Cloud Ops integration ensures excellent visibility. With Google Cloud Run, developers can package Functions with Google’s Functions Framework, deploy them using Knative, and Run them in mixed deployment scenarios. The platform also has improved support for programming languages compared to the 2020 evaluation.

Reference customers gave Google high marks for performance and simplicity, and praised Cloud Functions for security and scalability. In addition, the direct involvement and support of product management and customer support teams has been recognized by customers. Areas for improvement include better support for packaging third-party libraries with Functions (for example, for processing Kafka messages) and greater transparency of Cloud Functions runtime dependencies. If you want to deploy applications on a global scale and smooth the transition from functions to containers in a hybrid FaaS deployment scenario, Google Cloud Run should be the right choice for you.

  • IBM has perfected Cloud Functions and is ready to launch Code Engine. IBM Cloud Functions is based on the open source Apache OpenWhisk project. It adds built-in API gateways, good event flow support, and strong container-level function isolation. Based on the results of this evaluation, IBM is better than any other vendor in terms of service level agreement terms and availability. Cloud Functions, however, still needs improvement in some areas, such as limited network isolation options and support for reserved function instances. In our 2020 review, we noted the need for IBM to establish a unified function runtime strategy following its acquisition of Red Hat. The IBM Cloud Code Engine seems to be up to the task. While not yet in the general availability (GA) stage (and therefore not included in our analysis), Code Engine has added support for Kubernetes and Knative to IBM’s FaaS strategy.

IBM’s fast auto-scaling of function instances and high concurrency of computational workloads have been well received by reference customers. Other customers noted that support for OpenWhisk enables portability, which reduces the risk of vendor lock-in. Improvements include better choreography, observability, and support for reserved function instances. Existing IBM customers will find Cloud Functions ideal for integrating their workloads, and its extensive support for multiple programming languages can help most developers make the transition to a serverless architecture. The IBM Cloud Code Engine moving into the GA phase is also something to look forward to, especially for customers who are investing in Kubernetes-based infrastructure and want to incorporate functions into packaged objects.

competitors

  • Oracle prioritizes security, container, and event-driven integration. When we analyzed Oracle Cloud Functions in 2020, we viewed the platform as relatively new, and this was reflected in our evaluation. Oracle’s FaaS platform has been upgraded from the “Challenger” quadrant to the “competitor” quadrant in 2021. Oracle Cloud Functions is based on the open source Fn project, where Functions are packaged as OCI containers. Oracle built its FaaS platform with a focus on secure-by-design, which is what many corporate development teams want. By default, function instances are only available within the subnet of the customer’s virtual cloud network. Unlike some other FaaS vendors, Oracle does not charge extra for private cloud support. As a result, Oracle is steadily increasing adoption rates in North America, Europe, Asia and Latin America.

Reference customers gave Oracle Cloud Functions high ratings for performance, ease of use, development time, document depth, and overall execution cost. The disadvantage is the need to come up with an alternative solution to the occasional delay problem, which may be due to the current lack of support for reserved instances. Oracle Cloud Functions is ideal for enterprises that use other OCI services. In addition, the platform is now open to 23 geographies, enabling customers to deploy globally.

The challenger

  • Huawei’s FunctionGraph is just entering the market, but it’s promising. The functional workflow platform is still in its infancy, but we found during our evaluation that it has reached the market average with strong support for development tools, local development, and event-driven integration. The platform can execute functions that take up to 15 minutes to run and supports node.js, Java, Python, Go, C#, and PHP. As a newcomer to the market, the platform still has some shortcomings. For example, documentation is incomplete, support for special workloads is poor, and support for integration is inadequate. Availability is also limited in some areas. Therefore, we believe Huawei’s functional workflow platform is best suited for developers who want to deploy workloads in China, or in international markets such as Brazil, Chile, Mexico, Singapore, South Africa, or Thailand, where Huawei Cloud provides regional support. Huawei declined to participate in the full Forrester Wave™ evaluation process.

Overview of evaluation Report

We evaluate vendors strictly based on 40 criteria, which fall into three broad categories:

  • ** Current products. ** Each vendor’s position on the vertical axis of the Forrester Wave™ quadrant chart represents the performance of their current product features. Key criteria for evaluating these solutions include the developer experience, programming model, runtime execution environment, and security capabilities.
  • * strategic vision. ** The position on the horizontal axis represents the vendor’s strategic performance. We evaluated the FaaS platform providers’ vision, execution capabilities, ecosystems, engagement and pricing strategies.
  • ** Market share. ** is indicated by the size of the circle on the graph. Our market performance score reflects each vendor’s geographic distribution, customer base size and product revenue.

Vendor inclusion criteria

Forrester evaluated nine vendors: Alibaba, Amazon, Google, Huawei, IBM, Microsoft, Nimbella, Oracle, and Tencent. They all meet the following conditions:

  • Have FaaS platform. Each vendor has a FaaS platform that customers can readily use and run productive workloads.
  • Participated in Forrester’s Now Tech Serverless Architecture Vendor evaluation. The FaaS platform is a key component of building cloud-native applications with a serverless architecture. We detail the serverless architectures available to developers in Forrester’s Now Tech: Serverless Architectures, Q1 2021 report.

Supplementary material

Online resources

We organized all the Forrester Wave™ ratings and weights into an Excel file that included detailed product evaluations and supported custom ranking. Click the link at the beginning of the web version of the report on Forrester.com to download the file. We hope that these scores and default weights are only a starting point, and suggest that readers adjust the weights according to their personal needs.

Forrester Wave™ evaluation method

The Forrester Wave™ report provides guidance for organizations to purchase products in the technology marketplace. To ensure fairness to all participants, Forrester follows the Forrester Wave™ Methodology Guide to evaluate participating vendors. In this evaluation process, we conduct preliminary research and select vendors for subsequent evaluation. Then according to the inclusion criteria, further screening and determination of the final list. Detailed information on the product and strategy is then gathered through detailed questionnaires, presentations/presentations, and reference customer surveys/interviews. Finally, this information is combined with the analyst’s industry experience and expertise to rate a vendor through a relative scoring system that compares one vendor to other vendors participating in the evaluation. We clearly indicate the release date (year and quarter) of each Forrester Wave™ report in the title. For the evaluation of this Forrester Wave™ report, we used materials provided to us by the vendor prior to January 21, 2021, and did not accept additional information provided after that date. We want readers to analyze how the market and manufacturer’s products have changed over time. In accordance with the Forrester Wave™ and New Wave™ Vendor Audit Policy, Forrester requires vendors to review our estimates prior to release to ensure accuracy. Vendors flagged as not participating in the evaluation in the Forrester Wave™ quadrant meet the inclusion criteria as defined by us, but declined to participate or only partially participated in this evaluation. We rated these vendors according to the “Forrester Wave™ and Forrester New Wave™ non-participating and non-fully participating vendor policies” and placed them in a quadrant chart along with other vendors participating in the evaluation.

Rule of good faith

All of our research, including the Forrester Wave™ evaluation, follows the “Integrity Code” published on our website.

The original link

This article is the original content of Aliyun and shall not be reproduced without permission.