Introduction: If you are following the cloud Computing Conference these days, you must have noticed that Jiang Jiangwei (xiaoxie), alibaba partner and senior researcher of Alibaba Cloud, specifically mentioned a product called Grafana service in his speech at the Open Source Gravity Summit of Cloud Computing Conference, and specially spent a PPT to introduce this cooperation. What kind of product is it worth introducing?

The author | young jiaqi

In his speech at the Open Source Gravity Summit of The Cloud Computing Conference, Jiang Jiangwei (Xiaoxie), a partner of Alibaba and a senior researcher of Alibaba Cloud, specifically mentioned a product called Grafana service, and specially spent a PPT to introduce this cooperation. What kind of product is it worth introducing?

So, let’s talk about the Grafana service. Before we start, you can answer the following questions:

  • Are you addicted to the technology involved in buying machines, configuring networks, building environments, installing deployments, and preparing domain names and IP addresses?
  • Have you ever had a service that didn’t get up and couldn’t be accessed? SLA not guaranteed? Or can’t find the service IP address and password after a period of time?
  • Do you want to integrate the data of ARMS, SLS, Cloud monitoring, RDS, Trace and other cloud products into a unified large scale and have no idea where to start?
  • Do you want Grafana to connect to a private network in a VPC and provide email and SMS alerts and regular reports?
  • Have you installed various plug-ins and adjusted configuration parameters in a black screen on the command line? A fierce operation in the tiger, a look at the system heart block?

What is Grafana?

Grafana was born to solve these problems. Grafana is open source visualization and analysis software. It allows you to query, visualize, alert, and explore metrics wherever they are stored. In short, it provides tools to transform time Series database (TSDB) data into beautiful graphics and visualizations. You don’t know what Grafana is yet? So you must have seen the following visual kanban, which is configured in Grafana.

Why do Grafana hosting?

Grafana is a unified presentation solution for cloud native observability, covering all types of data sources and monitoring systems. There are tens of thousands of users on Aliyun who build Grafana themselves to complete the visualization of data. They also have to endure inconvenience in operation and maintenance deployment, personality setting, account management, Aliyun data source connection, alarm and other aspects when using Grafana.

In order to solve the various pain points of users in using Grafana, our cloud native team has been communicating with Grafana (Grafana Labs) for a year and reached a cooperation. Currently, the Grafana hosting service is in public beta (see the application address at the end of the article). This is Grafana’s first deep collaboration with a cloud vendor in the Asia Pacific region. Both parties will provide the first Grafana hosting service in China.

Grafana Labs and Ali Cloud will provide the first Grafana hosting service in China

What are we doing along the way?

Before deciding on the system architecture, we identified a number of concerns that users had as a Grafana managed service product. These included:

  • Ease of migration of self-built Grafana to managed Grafana (migration tool/configuration import);
  • Fully hosted automatic flexibility (free server operation and maintenance, high availability);
  • Support multi-party data sources (ARMS, Prometheus, SLS, user-built data sources) and alarm function integration;
  • Ali Cloud official data source pre-built data market;
  • Data access security (cloud account management/sub-account authorization/user group/user authorization);
  • Provide a variety of version selection, plug-in version selection, enterprise plug-in support;
  • Provides independent access to domain names/links.

Therefore, we took all of these factors into consideration when designing and selecting the system architecture to provide users with stable, easy-to-use, and powerful Grafana hosting services. Such as:

  • For better operation and maintenance management and cost consideration, managed service cluster adopts ACK cluster.
  • In order to ensure the data isolation of each user and the convenience of database upgrade, migration and maintenance, we adopt an independent database scheme (currently NAS SQLLite).
  • In order to connect the data in each VPC of customers and take performance, cost and supporting VPC scope into consideration, we adopt dual network cards +FC network connection.
  • Others, such as service state machine design, cloud service access, user authentication, authorization, independent domain name, are all considered and weighed;

As shown in the figure above, the middle layer hosts the client Grafana instance through the ACK cluster and connects to cloud service data sources, VPC data sources (FC channel) and other cloud service data sources below. Its upper layer provides Grafana service and console to the customer by interfacing with external services:

  • The Client uses (the service itself), Client -> Proxy -> Grafana
  • Monitor Management (Console), Client -> Console -> Grafana

The following figure shows the entire process for users to use Grafana: On the management and control side, the administrator or master account opens and buys Grafana workspace through the console, and authorizes other users or sub-accounts. At this point, we will create an independent Grafana Pod instance and a series of supporting resources for the user in the ACK cluster (e.g. NAS storage, configuration, self-monitoring, alarm integration, default plug-in, cloud service integration, etc.); From the perspective of the use and management of Grafana, the actual Grafana user or sub-account can log in to Grafana workspace and complete the configuration of data source and large panel normally. If the administrator has configured the VPC connection capability, the data source in the customer’s VPC can be accessed through our data Proxy.

What capabilities does the Grafana service provide?

1. Functional demonstration

After applying for open beta, we will review the application on a weekly basis and you will be able to experience Grafana hosting.

Each module is briefly described as follows:

  • Home page: The home page of the system will show the core advantages and functions of Grafana and the access steps.

  • Currently under ARMS domain name, independent domain name and entry will be provided later;

  • Create a Workspace: After entering the welcome page, click the “Create Workspace” button to create a Grafana hosting service

  • There is no charge to create a workspace during open beta, you simply need to fill out a form.

  • Version type: Only the standard version will be available during the public beta, and the enterprise version will be available later for your choice.

  • Version number: During the public beta, only 7.5.x version will be provided. Later, 8.1.x and other versions will be provided for your choice.

  • Password: indicates the login password of the admin account. You can use the Aliyun account to log in without the password.

  • Workspace list: After creation, you can manage your Grafana workspace

  • Work area details:

  • On the Workspace details page, you can view information about your workspace and click the “Login” button to use your Grafana directly.

  • Once you have the option of logging in via the “Alibaba Cloud” on the login page, you can start using the Grafana service.

  • Performance monitoring:

  • On the performance Monitoring page, you can view various data in your workspace (users, organizations, platters, resource consumption, request frequency and latency, etc.)

  • Parameter Settings:

  • Parameter Settings allow you to manage Grafana personalization Settings in a blank screen

  • For example, Oauth, service port, domain authentication sharing panel, and so on

  • Account Management:

  • On the Account Management page, you can reset the password of administrator Admin or authorize other accounts to access the Grafana instance.

  • The ability to invite users to join the Grafana workspace via SMTP will be added later.

  • Cloud Service Management:

  • Cloud service management features currently support synchronization of ARMS Prometheus data sources and marketplaces to Grafana workspaces.

  • In addition, SLS CMS supports data source configuration (see the help document for configuration methods).

  • In addition to supporting the above cloud service data sources, ARMS front-end monitoring and ARMS link monitoring data access are also being planned.

  • VPC data source channel management:

  • The VPC data source channel can be installed by specifying the VPC, switch, and security group. In this way, the VPC data source can be accessed from Grafana.

  • Access the data source in the VPC as if Grafana were currently deployed in the VPC (see the help on the page for details)

  • Alarm management:

  • Alarm management Integrates the ARMS alarm channel. Users can configure Alerting rules in Grafana by simply configuring contacts.

  • Plug-in management:

  • Plug-in management function provides convenient plug-in installation and uninstall functions, users do not need to care about the underlying commands and restart configuration adjustment and other tedious technical details;

2. Core strengths

  • Flexible, o&M free: With Grafana hosting services, you can use highly available services without managing o&M servers.
  • Unified visualization: support alicloud data sources (such as ARMS, Alicloud Prometheus monitoring, SLS, etc.), self-built data sources and third-party cloud vendor data sources (cloud monitoring, link monitoring, front-end monitoring data will be supported later).
  • Data security and authorization: Support SSO of Ali Cloud account and self-established account system, and realize fine management of data source and market as well as data backup and recovery (under development).
  • Data access across Multiple VPCS: The same Grafana workspace can connect to data sources in multiple private VPCS over a private network.

3. Functions and features

  • Default Integrated cloud services The default integrated cloud services, such as ARMS, Prometheus monitoring, Cloud monitoring, SLS, and Elasticsearch, provide data source configuration, preset market, and one-click alarm of various cloud services.
  • You can use the Grafana plug-in to connect your tools to your team. The data source plug-in connects to existing data sources through apis and renders the data in real time without requiring you to manually retrieve or migrate the data.
  • Custom Alarm System using Grafana Alarms, you can create, manage, and mute all alarms in a simple UI, allowing you to easily integrate and centralize all alarms (some features under development).
  • Multidimensional data queries support cross-data source queries, data source renaming, aggregation, composition, and computation.
  • Self-created data sources You can connect multiple VPCS in the same region or add data sources from multiple VPCS to the same workspace, and query and display alarms in a unified manner.
  • The panel editor makes it easy to configure, customize, and browse all panels through a consistent UI to set data options across all visual panels.

The Grafana hosting service will help you solve the pain points of using Grafana and make it easier to use.

The original link

This article is the original content of Aliyun and shall not be reproduced without permission.