Audience gentlemen, try this data visualization background with Vue masturbation

Project Github address, welcome Star

✨ ✨ dongsuo/vue – data – board ✨ ✨

Let’s start with an online address to get a feel for it (it may be a bit slow because you’re using leanCloud’s free package on the back end) :

vue-data-board

P.S. recommends that you create your own account as much as possible (you can fill in any password you like). If you’re using the default test account, don’t change anything because it won’t be read by anyone else, because any changes you make will be saved to the backend database.

A few more gifs to catch the eye:

Drag and drop enables data query and visualization

Add charts to kanban

Customize kanban layout by dragging and dropping

origin

Somehow, there is a growing need for data analysis and data visualization in companies. These requests come from data analysts (if any), product, operations, and development, middle and senior management, from exporting data from a single table, and from analyzing and visualizing data from multiple dimensions. Also hopes to integrate multiple visual chart form in a page visual statements, have a plenty of one-off demand, have a plenty of cyclical normalized requirements, these requirements are often disorganized, and a lot of dimension data, interaction is more complex, various parameters of visual chart is numerous, it’s hard to do, And a lot of data is more sensitive and need strict authority control, a long time, authority management has become a headache, all kinds of factors lead to a variety of data needs to spend a lot of energy in the front-end development, the amount of hair is getting thinner.

Believes that many companies will have similar data requirements, scattered but must, though there are many products on the market provides similar data management function, but due to the sensitivity of the data, many companies are reluctant to put their own data in other platforms, and data management of these products are expensive and difficult to custom function. So many companies in the data requirements will fall to the front, although these demand scattered even temporarily, but it’s not easy to implement, the visual effect of visual need to configure various parameters, and the large amount of data to verify, clean, transform, and makes the docking interface debugging and become very trouble, increase the workload of the front end, It also makes front-end coding very annoying.

At this time to do a powerful flexible and easy-to-use data analysis background is very useful, through the foolishly drag and drop to generate charts, can let colleagues without data basis can quickly get started; By integrating the company’s data sources together, the data can be easily managed; By building kanban to integrate multiple charts, you can quickly generate reports, easy to view and share; By customizing chart parameters, the pain of front-end parameter tuning can be effectively eliminated. (A perfectly organized parallel sentence, Oh yeah!)

I build in our data analysis of the background also nearly a year, in fact, I started at the first internship contact BI, data related to things, in the process of data analysis of the background of building also stepped on many pits, experienced colleagues during the period of all kinds of fun, try a variety of solutions, today although the ability see how much code, But I know a lot about the chaos and vagaries of business requirements.

After more than half a year of pain, I according to business requirements, reference other data analysis product, set up a new data analysis platform, this platform now in our company is in good working order, has met the demand part of the data on the one hand, on the other hand also has good expansibility, can deal with more complex business scenarios. I took the front end out and formed this open source project:

dongsuo/vue-data-board

This project is mainly built with Vue+ElementUI. The visualization part is echarts. The drag and drop part is vuE-Grid-Layout and vueDraggable libraries. Some of the technology, ideas, and some of the auxiliary code are based on pan’s background project:

PanJiaChen/vue-element-admin

The back-end of the project is built with [leanCloud](http://leancloud.cn), except for a few mock data. The Demo data database uses a foreign Free database, Free MySQL Hosting, which is slow and does not have root permission, but the test is enough. Other user data, charts, kanban, etc., are stored by leanCloud object.

Key function points and implementation principles

Firstly, the main function points and implementation principles of the project are introduced:

1. Drag and drop fields to query data

This function is mainly used to build SQL statements to query databases. In the back end, Presto is used to query various databases in a unified manner. In addition to traditional relational databases, IT can also query HDFS. Front-end part of the data query is mainly through the deconstruction of SQL statements, divided into dimensions, data, screening, sorting, number of data and other interactive elements, easy to use without SQL based users. The user’s interaction in the front end dynamically generates SQL according to specific grammar rules and sends it to the back end for database query. Of course, the current SQL build function is not complete, there are some SQL syntax still need to support, has been included in the development plan.

2. Perform visual rendering of data

Visual rendering of data is mainly to judge the available chart types by the dimensions and values of data query, and then use the dynamic components of vue.js to render the corresponding chart components:

/ / the Vue. Js dynamic component rendering the corresponding chart type < component: is = "currentType.com ponentName" : data = "chartData" : schema = "schema" :chart-style="chartStyle" class="visualize-window" />Copy the code

Different data are suitable to be displayed in different charts. For example, the data of pie chart is different from that of stacked bar chart. Therefore, the data structure required by each chart needs to be defined:

MatchRule: {desc: '1 dimension 1 value; IsUsable (dimensions, calculs) { return (dimensions.length === 1 && calculs.length === 1) || (dimensions.length === 0 && calculs.length >= 1) }},Copy the code

It then determines whether the chart type is available based on the matching rules.

3. Save and echo charts

After the front end generates the chart, it can be saved to the back end. Because there are too many fields needed to define a chart, and these fields may be changed frequently, so it is impossible for the back end to define these fields in the database one by one. Therefore, we adopt the scheme that the front end maintains these fields. The backend stores all content in a database field, such as Content, as a string or JSON object.

This way, there is no problem with the graph echo and the content field should be resolved according to the logic used to generate the graph.

4. Integrate charts into kanban

Many times a business needs to view multiple charts at the same time, and a Kanban is needed to integrate multiple charts. Kanban actually refers to Dashboard, and there is no proper Chinese translation for it, so kanban barely makes sense.

Kanban integration of multiple charts is just a relational relationship management in the back end, and the front end needs to layout the page according to the kanban associated charts, and then query data and visual rendering according to the saved charts. Page layout is mainly the use of VUe-grid-Layout grid layout, but also support drag and drop and size adjustment. The logic of visualization is the same as visualization when creating a chart, and I won’t go over it again.

The problem here is how to calculate the position of the new chart when adding a new chart to an existing layout kanban. In fact, this is similar to the problem of waterfall flow in the picture, but since each item is of variable width and height, it is actually more difficult. I have found the idea and made some calculations, but it is not perfect yet. I will write an article to introduce it after completion.

5. Data permission problem

In fact, the company’s data is quite sensitive. For listed companies, data will affect the stock price trend, and for non-listed companies, it will affect the progress of investment and financing, which are related to the company’s financial and even life and death events. Therefore, the authority management of data is very important. In this case, the solution for this part of the project was not simple, but the complexity lay in the back end rather than the front end. To put it simply, our authority is achieved at the level of data table. The administrator adds data source to the system and defines the scope of authority of the data source at the same time, such as product, operation, development and other authority roles. After users log in to the system, the administrator assigns rights and roles to users. Users can query only the data that can be viewed by their own roles. Permissions and data source management have not yet been added to the open source project, so I will not go into details. I will dig a hole and fill it in later.

6. Other technical points

• The first project was built using [email protected]. In addition to the default configuration, some custom configurations were made through vue.config.js. In addition, there is also the configuration of environment variables, etc. For the configuration of the project, please refer to this article by Pan:

  Hands touch hands, take you with vUE Slapping backstage series five (v4.0 new version) – Nuggets

• Many ICONS have been used in the project. Although the icon of Element UI has been increased a lot after version 2.8.x, it still cannot meet our needs (some commonly used ICONS are still missing, such as saving), so we need to solve the problem of icon by ourselves. Thanks to The designer At IconFont for this beautiful library of data visualizations. As for how Icon manages its use with grace, here is pan’s article again:

  Hand to hand, with your elegant use of icon – nuggets

• I used Postman to manage the back-end interface document of the project. In fact, Postman has many powerful functions, not only an interface testing tool, but also interface document management. I usually start front-end and back-end projects locally at the same time during development. During local development, I judge the address of the back-end interface connected locally by environment variables:

    const fetchInstance = axios.create({
    	baseURL: process.env.VUE_APP_BASE_API
    *}*)
  Copy the code

  After local development, the front end is packaged and published to the GH-Page branch (Git subtree push — Prefix Docs Origin GH-Pages), and the back end is deployed through the CLI tool provided by leanCloud with one click, which is quite convenient. The packaged publishing of the front end can actually be configured with Travis- CI for automatic deployment, which I haven’t done yet (for the procrastinators who have abandoned therapy, here).

• Project login authorization, chart and kanban add, delete, change and check are all solutions provided by LeanCloud. In fact, using the JS-SDK of LeanCloud, the front-end can also easily implement the add, delete, change and check of object storage without writing the back-end interface. But in order to keep the project code pure and avoid introducing strange things like av.query into the code, I still made the back-end part by myself. Of course, this part is relatively simple, after all, it is only a Demo, mainly based on koa.js to do some increase, delete, change and check, basically oriented to document programming.

• Vuex is used for state management in the project, but there is not much need for global state management so far, only user tokens are stored in the store. In addition to state management, I also tried Vue’s simple state management mode (code here) in the charting section of this project, which was described in an article on Nuggets:

  Struggling to manage complex component state? Try vue simple state management Store mode – Nuggets

  This pattern works fine for complex components in large projects, and easily solves the problem of state sharing within complex components. However, it is not particularly suitable for my project at present, because the state management of this part is also quite appropriate in the global state. Of course, I have different opinions here, and I think it is good for the present.