In the presentation optimization of tabular data, on-demand rendering of tabular data presentation is mentioned. Instead of rendering the long list in its entirety, it displays a portion of the long list data based on the height of the container element and the height of the list item element to improve the performance of infinite scrolling. The implementation of the display on demand scheme is the virtual list mentioned in the title of this article.

There are many ways to implement virtual list. This paper analyzes the react-virtual-list component

What is a virtual list?

Before we get started, a brief definition of the virtual list.

According to the above, virtual list is an implementation of the idea of on-demand display. That is, virtual list is a technique that renders part of a long list of data according to the visual area of the scrolling container elements.

In short, a virtual list is a list of “visual area renders”. There are three concepts to understand:

  • Scroll container elements: In general, the scroll container element iswindowObject. However, we can specify any one or more scroll container elements on a page through layout. Any element that produces horizontal or vertical scrolling internally is a scroll container element considering that each list item simply renders plain text. In this article, only vertical scrolling of elements is discussed.
  • Scrollable region: Scrolls the inner content area of the container element. If you have 100 entries and each list item is 50 in height, then the height of the scrollable area is 100 * 50. The current height of the scrollable region can be passed through the element’sscrollHeightProperty acquisition. The user can scroll to change the display portion of the list in the viewable area.
  • Viewing area: Visually visible area of the scroll container element. If the container element iswindowThe viewable area is the viewport size of the browser (i.eVisual viewport); If the container element is somedivElement, whose height is 300, has a vertical scroll bar to scroll to the right, so the visually visible area is the visible area.

To implement a virtual list is to change the rendering part of the list in the visual area when dealing with user scrolling. The specific steps are as follows:

  • StartIndex to calculate the start data of the current visible region
  • Computes the endIndex of the current visible region end data
  • Calculates the data for the current visible area and renders it to the page
  • Calculate the startIndex data offset startOffset in the entire list and set it to the list
  • Calculate endOffset, the offset position of endIndex data relative to the bottom of scrollable area, and set it to the list

Please refer to the following figure to understand the above steps:

Element L refers to the last element in the current list

As can be seen from the figure above, startOffset and endOffset will expand the height of the contents of the container elements so that they can be rolled sustainably. In addition, it keeps the scrollbar in the correct position.

Why a virtual list?

Virtual lists are an optimization for long lists. In front end development, you’ll encounter business types that don’t use pagination to load list data, which we call long lists. For example, in some foreign exchange trading systems, the front end will quasi-real-time display of the user’s position (gains, losses, hands, etc.), at this time for the user’s position list is generally not pagination.

In this article, we define a long list as one that has a data length greater than 999 and cannot be presented in a paginated form.

How long does it take to render a long list completely without optimizing it? A simple demo will be written to test the following.

Test environments: Macbook Pro(Core I7 2.2g, 16G), Chrome 69, React 16.4.1

In the demo, let’s test how long it takes the browser to render 10000 simple nodes:

import React from 'react'

const count = 10000

function createMarkup (doms) {
  return doms.length ? { __html: doms.join(' ') } : { __html: '' }
}

export default class DOM extends React.Component {
  constructor (props) {
    super(props)
    this.state = {
      simpleDOMs: []
    }

    this.onCreateSimpleDOMs = this.onCreateSimpleDOMs.bind(this)
  }

  onCreateSimpleDOMs () {
    const array = []

    for (var i = 0; i < count; i++) {
      array.push('<div>' + i + '</div>')
    }

    this.setState({
      simpleDOMs: array
    })
  }

  render () {
    return (
      <div style={{ marginLeft: '10px' }}>
        <h3>Creat large of DOMs：</h3>
        <button onClick={this.onCreateSimpleDOMs}>Create Simple DOMs</button>
        <div dangerouslySetInnerHTML={createMarkup(this.state.simpleDOMs)} />
      </div>
    )
  }
}Copy the code

When a Button is clicked, onCreateSimpleDOMs is called to create 10,000 simple nodes. You can see the following data from Chrome’s Performance TAB:

As you can see from the image above, it took about 693ms to go from Event Click to Paint. The main time consumption for rendering is as follows:

  • Recalculate Style: 40.80 ms
  • Layout: 518.55 ms
  • Update Layer Tree: 11.84ms

During the Recalculate Style and Layout phases, the ReactDOM calls the setInnerHTML method, which adds the created HTML fragment to the corresponding node primarily through the innerHTML method

Then, we create 10,000 slightly more complex nodes. Modify components as follows:

import React from 'react' function createMarkup (doms) { return doms.length ? { __html: doms.join(' ') } : { __html: '' } } export default class DOM extends React.Component { constructor (props) { super(props) this.state = { complexDOMs:  [] } this.onCreateComplexDOMs = this.onCreateComplexDOMs.bind(this) } onCreateComplexDOMs () { const array = [] for (var i = 0; i < 5000; i++) { array.push(` <div class='list-item'> <p>#${i} eligendi voluptatem quisquam</p> <p>Modi autem fugiat maiores. Doloremque est sed quis qui nobis. Accusamus dolorem aspernatur sed rem.</p> </div> `) } this.setState({ complexDOMs: array }) } render () { return ( <div style={{ marginLeft: '10px'}}> <h3>Creat large of DOMs: </h3> <button onClick={this.onCreateComplexDOMs}>Create Complex DOMs</button> <div dangerouslySetInnerHTML={createMarkup(this.state.complexDOMs)} /> </div> ) } }Copy the code

OnCreateComplexDOMs is called when Button is clicked. You can see the following data from Chrome’s Performance TAB:

As you can see from the image above, it took about 964.2ms to go from Event Click to Paint. The main time consumption for rendering is as follows:

  • Recalculate Style: 117.07 ms
  • Layout: 538.00 ms
  • Update Layer Tree: 31.15ms

For each of the above tests for 5 times, and then take the average of each indicator, the statistical results are as follows:

– Recalculate Style Layout Update Layer Tree Total
Render simple nodes 199.66 ms 523.72 ms 12.572 ms 735.952 ms
Render complex nodes 114.684 ms 806.05 ms 31.328 ms 952.512 ms
  1. Total = Recalculate Style + Layout + Update Layer Tree
  2. The demo test code is: test code

As can be seen from the above test results, it takes 700ms+ to render 10000 nodes. In the actual business, each node of the list needs about 20 nodes, and the Layout will be much more complicated, and it will take longer in the Recalculate Style and Layout stages. Then, 700ms can only render about 300 ~ 500 list items, so the complete long list rendering is basically difficult to meet the requirements of the business. Instead of a full long list, there are generally two ways to render: on-demand and lazy (lazy). The common infinite scrolling is an implementation of deferred rendering, while virtual lists are an implementation of on-demand rendering.

Delayed rendering is beyond the scope of this article. Next, this article will briefly introduce a virtual list implementation.

implementation

This section will create a VirtualizedList component and walk through the implementation of the virtual list with the code.

For simplicity, we set the window as a scrolling container element, add style rules for both HTML and body elements height: 100%, and set the viewable area to the browser window size. VirtualizedList will reference the mobile side of Twitter in the DOM element layout:

class VirtualizedList extends Component { constructor (props) { super(props) this.state = { startOffset: 0, endOffset: 0, visibleData: [] } this.data = new Array(1000).fill(true) this.startIndex = 0 this.endIndex = 0 this.scrollTop = 0 } render () { const  {startOffset, endOffset} = this.state return ( <div className='wrapper'> <div style={{ paddingTop: `${startOffset}px`, paddingBottom: `${endOffset}px` }}> { // render list } </div> </div> ) } }Copy the code

In the implementation of virtual list, there are also two cases: list items are fixed height and list items are dynamic height.

List items are of fixed height

Since list items are of fixed height, the convention is that each list item is 60 in height and the list data is 1000 in length.

First, we estimate the number of elements that can be rendered in the viewable area based on its height:

const height = 60
const bufferSize = 5
// ...

this.visibleCount = Math.ceil(window.clientHeight / height)Copy the code

Next, calculate startIndex and endIndex and initialize the data that needs to be rendered for the first time:

/ /... updateVisibleData (scrollTop) { const visibleData = this.data.slice(this.startIndex, this.endIndex) const endOffset = (this.data.length - this.endIndex) * height this.setState({ startOffset: 0, endOffset, VisibleData})} componentDidMount () {this.visiblecount = math.ceil (window.innerheight/height) + bufferSize this.endIndex = this.startIndex + this.visibleCount this.updateVisibleData() }Copy the code

As mentioned above, endOffset is to calculate the offset position of data corresponding to endIndex relative to the bottom of scrollable region. In this demo, the height of the scrollable region is 1000 * 60, so the offset of the endIndex data from the bottom is (1000 – endIndex) * 60.

The initial value of startOffset is 0 because it is initializing the data that needs to be rendered for the first time.

In order to calculate the visible area, you just need to calculate startIndex, because visibleCount is a fixed value and bufferSize is a buffer value, which increases the buffer area and makes it less jarring at normal sliding speed. And the value of endIndex is equal to startIndex plus visibleCount; Also, when the user scrolls to change the data in the visible area, the value of startOffset needs to be calculated to ensure that the new data will appear in the user’s browser viewport:

If the startOffset value is not computed, elements that should be rendered in the viewable region will be rendered out of the viewable region. As you can see from the figure above, the value of startOffset is the offset from the top border of element 8 (the uppermost element in the viewable area) to the top border of element 1. Element 8 is called the anchor element, which is the first element in the region. Therefore, we need to define a variable to cache some of the location of the anchor element, as well as the location of the rendered element:

/ /... This. cache = [] this.cache = {index: 0, this.anchorItem = {index: 0, 0, // The offset of the top of the anchor element from the top of the first element (i.e., startOffset) bottom: 0 // the offset of the bottom of the anchor element from the top of the first element} //... cachePosition (node, index) { const rect = node.getBoundingClientRect() const top = rect.top + window.pageYOffset this.cache.push({ index, top, bottom: top + height }) } // ...Copy the code

The cachePosition method is called after each component is rendered (componentDidMount), where node is the corresponding component node element and index is the node’s index:

// Item.jsx

// ...
componentDidMount () {
  this.props.cachePosition(this.node, this.props.index)
}

render () {
  /* eslint-disable-next-line */
  const {index} = this.props

  return (
    <div className='list-item' ref={node => { this.node = node }}>
      <p>#${index} eligendi voluptatem quisquam</p>
      <p>Modi autem fugiat maiores. Doloremque est sed quis qui nobis. Accusamus dolorem aspernatur sed rem.</p>
    </div>
  )
}
// ...Copy the code

Once the anchor element and rendered element positions are cached, the user’s scrolling behavior can then be handled. Take the user scrolling down (the direction in which the scrollTop value increases) as an example:

/ /... / / computing startIndex and endIndex updateBoundaryIndex (scrollTop) {scrollTop = scrollTop | | 0 / / the user under normal rolling, Const anchorItem = this.cache.find(item => item.bottom >= scrollTop) this.anchoritem = {... AnchorItem} this.startIndex = this. anchoritem. index this.endIndex = this.startIndex + this.visibleCount} // Scroll the event handler handleScroll (e) { if (! This. Doc) {/ / compatible with iOS Safari/Webview this. Doc = window. The document. The body. The scrollTop? window.document.body : window.document.documentElement } const scrollTop = this.doc.scrollTop if (scrollTop > this.scrollTop) { if (scrollTop >  this.anchorItem.bottom) { this.updateBoundaryIndex(scrollTop) this.updateVisibleData() } } else if (scrollTop < This. ScrollTop) {// Scroll up (' scrollTop '= scrollTop)} this.scrollTop = scrollTop} //...Copy the code

In the scroll event handler, startIndex, endIndex, and the position of the new anchor element are updated (i.e., startOffset is updated), and then the render data for the visual area can be dynamically updated:

Complete code in can poke: fixed height virtual list implementation

List items are dynamically high

In this case, the implementation idea is pretty much the same as the list item height. The minor difference is how to get the exact height of a list item when caching its location information. We’ll start by changing part of the logic of cachePosition:

/ /... cachePosition (node, index) { const rect = node.getBoundingClientRect() const top = rect.top + window.pageYOffset this.cache.push({ index, Top, bottom: top + rect.height})} //...Copy the code

How do you calculate visibleCount since the height of list items is not fixed? Let’s think about each list item and just render some plain text. In a real project, some list items may have only one line of text, and some may have multiple lines of text. In this case, we will give the list item an estimated height based on the actual situation of the project: estimatedItemHeight.

For example, if we have a long list to render the user’s article summary and specify that the summary should display no more than three lines, we take the average height of the top 10 list items of the list as the estimated height. Of course, we can expand the sample size to make the height estimate more accurate.

Now that we have the estimated height, we can calculate visibleCount by replacing the height in the original code with estimatedItemHeight:

/ /... const estimatedItemHeight = 80 // ... This.visiblecount = math.ceil (window.innerheight/estimatedItemHeight) + bufferSize //...Copy the code

We create some random data with faker.js and assign it to data:

// ...
function fakerData () {
  const a = []
  for (let i = 0; i < 1000; i++) {
    a.push({
      id: i,
      words: faker.lorem.words(),
      paragraphs: faker.lorem.sentences()
    })
  }

  return a
}
// ...

this.data = fakerData()

// ...Copy the code

Modify the render logic for the list item, otherwise unchanged:

// Item.jsx

// ...

render () {
  /* eslint-disable-next-line */
  const {index, item} = this.props

  return (
    <div className='list-item' style={{ height: 'auto' }} ref={node => { this.node = node }}>
      <p>#${index} {item.words}</p>
      <p>{item.paragraphs}</p>
    </div>
  )
}
// ...Copy the code

At this point, the height of the list item is already dynamic, and based on the actual rendering situation, we give an estimated height of 80:

Complete code in can poke: dynamic height of virtual list implementation

What if the list item isn’t rendered plain text? For example, when you call cachePosition for Item componentDidMount, do you get the height of the corresponding node correctly? In the case of rendering graphics, there is no guarantee that the image will be rendered when the list item component is mounted (componentDidMount), and the height of the corresponding node is not accurate, so when the user scrolls to change the rendered data for the visible area, It is possible to have overlapping elements:

In this case, we can get the correct height if we can listen for the size of the Item component node. ResizeObserver, which at the time of this writing is only supported by Chrome 67 and above and is available in all major browsers, might be just the thing. Here are some information I collected for your reference (bring your own ladder) :

  • ResizeObserver: It’s Like document.onresize for Elements
  • ResizeObserver
  • caniuse#resizeobserver

conclusion

In this paper, first of all, a simple definition of the virtual list, and then from the perspective of the long list of why the need for a virtual list, and finally on the list item fixed height and fixed height of two scenarios with a simple demo detailed virtual list implementation ideas.

In the scenario where the list item is dynamic height, the rendering of plain text and text mixed scene is analyzed. The former provides a specific demo, and the latter provides a reference ResizeObserver scheme for how to monitor changes in element size. Based on ResizeObserver, I also implemented a react-virtual-list component that supports blending of rendered text and text (as well as plain text) for your reference.

Of course, this isn’t the only way to implement virtual lists. In the implementation of the react-virtual-list component, I also read the source code of different virtual list components, such as: Then I’m going to look at them all from a source code perspective in the next series of articles.

reference

  • Complexities of an Infinite Scroller
  • Infinite List and React
  • Talk about long lists in front-end development
  • Talk about the realization of the front end virtual list