First, BI visualization large screen common technology

It is the first time for me to contact visual development, so I have investigated and experienced many mainstream visual open source solutions in the development process, and selected some of them to be applied to our project, here to share with you

1. Main functions of the project

Through the analysis of product requirements, we found that the visual requirements of our project mainly include: visual chart display; Chart support drag, zoom; Rich text editing; Json edit display editor; Screenshots. Canvas layout needs these aspects. The core function diagram is as follows:

The functions we need to implement in our project are also some of the most common functions in visual development, so I would like to share with you some useful open source tools.

1.1 every

Apache EChartsTM is a top-level project of the Apache Software Foundation (ASF). It supports rich chart types, tens of millions of data visualization rendering capabilities, and an active community. Development of the document

When choosing the visual library, I used Antv’s G2Plot through Ant Design Pro, and then encountered a lot of pits. Many of them gave up before they could find a solution. I have to say that the community of G2Plot still has a big gap with Echart.

1.2 html2canvas

Html2canvas is a screenshot plug-in, it can easily help you to convert HTML code into Canvas, and then generate images that can be saved and shared.

Html2canvas reference document

1.3 the react – the grid – layout

The React library is very useful for visual diagram drag/zoom layout. In our project, it is used for kanban diagram drag/zoom layout with simple configuration.

Reference documentation

1.4 ali – react – table

A high-performance React table component from Ali supports crosshatches and perspectives, with built-in virtual scrolling, which automatically turns on when there is a large amount of data. Rendering is very smooth, the author is very nice, and the issue is almost immediately responded, highly recommended. (PS: sloppy documentation)

Making the document

1.5 codemirror/react – codemirror2

Codemirror is an online code editor tool that can be used for real-time code highlighting, data source management, SQL input editor, etc.

The document

1.6 the react – DND

React – DND is a set of React high-order components created by React and Redux core author Dan Abramov to help build complex drag-and-drop interfaces while keeping components separate.

The document

1.7

Idlize

The document

Translation code optimization strategy – Idle Until Urgent

1.8 jsoneditor/jsoneditor – react

Very useful JSON editor

The document

1.9 puppeteer

Puppeteer is the Google Chrome team’s official Headless Chrome tool. Puppeteer is a Node library that provides an advanced API to control the DevTools protocol on the Headless Chrome.

Chinese document

The front-end architecture and defects of V1.0

1. The react architecture

The previous projects used VUE, and there were many problems, such as no restrictions on development specifications; Packaging, lack of automation of publishing operations, poor layering of code within projects, and so on. For the new project, we need to redesign the front-end architecture and migrate from VUE to React.

I made a general drawing of our project architecture. Since our resources are limited, we will try our best to build with existing resources. The react project was built on official scaffolding, with code specification, format verification, commit-log submission specification, automated publishing, local debug line interface for problem location, and optimized code layering.

2. Main functional modules

There are three main functional modules, namely multidimensional analysis page, data set page and configuration page. The most complex one is the configuration page, which contains the configuration of various Echart charts, the configuration of a single chart, the configuration of indicators (dimensions) and so on. The chart preview area of the multidimensional analysis page is fully reusable.

3. The defect

  • Common component design is missing or badly designed

    Version 1.0 time very, very tight (colleagues, one needs to be done a month online version 1.0), so that the whole project is not very good to design public components, such as indicators for each chart configuration panel/dimension/filter/time screening component (chart preview the multidimensional analysis section), or not to extract common components, Or the common components are poorly designed because there is no time…

  • The front end lacks error monitoring

    Lack of monitoring alerts for errors that cause page crashes

  • Lack of front-end performance monitoring

    Lack of monitoring of page loading performance

Iii. Upgrade and improve the front-end architecture of V2.0 version

In version 1.0, due to the development period problems, component design, front-end monitoring and other aspects have great defects, but because version 1.0 only supports indicator card/table/line chart these three charts, does not support a series of functions such as latitude, indicators, sorting, these defects have not shown a great impact.

In the 2.0 iteration, we need to add a variety of chart types, and add a series of functions such as sorting, aggregation, and customization to dimensions and indicators. If we do not fix the defects of 1.0, it will cause our code to become extremely bloated and difficult to maintain. There is no error monitoring alarm, and it is difficult for us to actively locate bugs on line. Therefore, during the iteration of the release, we fixed each of these defects.

1. Common component design

First step, we need to design common components in the configuration panel, such as indicators, dimensions, time filters, filters, etc. Components, indicators and dimension components are roughly as follows:

The react of components at the time of design need to try to design a smooth props, does not contain side effects, props default values are given as much as possible, single responsibility principle to design, such as, for example, the dimensions and indicators will support a list, according to the development progress, in different diagrams can show only part of the configuration, such as illustrations, belongs to the dimension, And only the display name can be set, so we can design a listConfig as props to indicate which parts need to be shown.

Design appropriate React common components for different functions and upgrade our architecture.

2. Front-end monitoring

Adding error and performance monitoring to a project is an important means to ensure smooth operation of the project

2.1 Error Monitoring

Error monitoring is mainly carried out from the following four aspects

  • React uses the boundary error catching method componentDidCatch provided by React

  • Use sourceMap to locate error source code

  • Nailing alarm

  • Error message friendly interface

The source code is also relatively simple

import { sendPageErrLog } from '@/const/host' import StackTrace from 'stacktrace-js' import { Empty } from 'antd' import  ErrorImage from '@/static/images/cry.png' export default class ErrorBoundary extends React.Component { state = { error:  null, errorInfo: null } componentDidCatch(error: Error, info: ErrorInfo) { this.setState({ hasError: true, error: error, errorInfo: info }) let urls = window.location.href if(urls.includes('local')) return false let params: any = { errUrl: urls.split('#')[1], serverHost: urls.split('#')[0], errMessage: error.message } StackTrace.fromError(error).then((err) => { params = { ... params, ... err[0] } window.request.post(sendPageErrLog, params).then((res: any) => {}) }) /* do other thing */ } handleClick = () => { location.reload() } render() { if (this.state.errorInfo) { return ( <Empty image={ErrorImage} imageStyle={{ height: 60}} description={<span> Data load failed <a style= {{color:'#1890ff'}} href="javascript:void(0); </a> </span> </Empty>)} return this.props. Children}}Copy the code
2.2 Performance Monitoring

Because there is no public monitoring platform, the project simply added some key data monitoring, performance monitoring indicators need to be gradually improved.

At present, performance monitoring is mainly to monitor the loading speed of kanban and ensure the smoothness of the page, mainly from the following aspects:

  • Use Idlize, idle operation

  • Calculate the loading time of the first screen kanban

  • Calculates the loading time for the entire kanban

Loading time of first screen Kanban: First, we need to calculate the number of first screen kanban, because the chart in kanban supports zooming and dragging, so this is troublesome, but because we have a limit on the number of kanban in a row, we can calculate by exhaustive exhaustive

let clientHeight let temp = document.querySelectorAll('.react-grid-item') let resultHeight = [] let resultWidth = [] let  cards = 0 if(temp.length ! = = 0) {clientHeight = document. Body. / / minus the tabbar clientHeight - 90 for (let I = 0; i<temp.length; i++){ let height = temp[i] && temp[i].getBoundingClientRect().height-20 //margin resultHeight.push(height) resultWidth.push(layoutList[i].w) } let i = 0 for(; i<resultWidth.length;) { if(resultWidth[i] == 12){ cards++ clientHeight = clientHeight-resultHeight[i] if(clientHeight < 0) break i++ } if(resultWidth[i]+resultWidth[i+1] == 12){ cards = cards + 2 clientHeight = clientHeight-resultHeight[i] if(clientHeight  < 0) break i = i+2 } if(resultWidth[i]+resultWidth[i+1]+resultWidth[i+2] == 12){ cards = cards + 3 clientHeight = clientHeight-resultHeight[i] if(clientHeight < 0) break i = i+3 } } } props.mobileSet.setFirstContentCard(cards) console.log(cards) },[layoutList.length])Copy the code

Calculation steps: 1. First count the width and height of all charts in the current Kanban configuration resultWidth and resultHeight 2. Then set the number of first screen kanban to cards. 3. Perform exhaustive selection of three kanban, two kanban and one kanban in a row to calculate the number of first screen kanban

How to calculate kanban loading time?

The main query interface query request completion time is used to count the loading time of kanban. We set an apiTime array in store. After each chart in Kanban calls query interface, push a value to the apiTime. Set the statistics of the loading time of the first screen chart when the apiTime length reaches the number of the first screen kanban on the main page, and the statistics of the kanban loading time when the apiTime length reaches the total number of the first screen chart.

3. Develop customized projects based on configured projects

The thing is, in the last quarter, we have a requirement is a customized development flow analysis platform, mainly to curl orders from advertising to the users of the whole link traffic data to the platform, the main function is to achieve a flow overview page, a list of new pages, allows users to new screening (global) to customize their own traffic kanban, etc., It sounds like a big project, but it only took half a month from review to launch.

Since we need to launch a project in a very short time, the way we adopt is to rely on the existing configuration project to develop a customized project. First, we need to evaluate the differences between the two, and then use the existing part to support the customized development to minimize the development work.

What are some examples of differentiation and how did we deal with it?

  • Configuration kanban does not have global filtering, and is queried in the unit of charts. And custom kanban is querying in units of multiple chart kanban. There’s no good way to do this, interface queries from single chart –> promise.all(),

  • Customization has multiple charts that are not configured to build a new type of chart using the data structure of the most similar configured chart

  • Customization is a set of index control linkage polymerization and polymerization not double chart Configuration to make use of the index card configuration all index aggregate data, using bar charts single configuration index aggregate data, each switch indicator, not the aggregated data to request to show the new data interface, passed under the style of front-end control the index aggregate data aggregated data show.

  • The overview page data query interface is slow and the back end uses caching and bitmaps to speed things up