The cause of
When I was looking at the interview questions, I suddenly saw how to process 100,000 pieces of data on the front end, and the user interaction was displayed in the form of a drop-down list. I thought it was interesting, so I started to do it myself
Interviewer’s test point
I analyzed a wave, the interviewer may test the following points:
- Front-end render massive DOM nodes created with fragement?
- The browser is bound to be stuck with the massive DOM nodes. How can users interact with each other without knowing it
- How to connect 100,000 pieces of data, all in memory?
Virtual list
The first thing that comes to mind is the virtual list technology, assuming that in our window, we can display a maximum of 20 data, we only need to create 20 DOM nodes, as long as the user mouse scroll to update the DOM content, the effect is achieved, direct action!!
Virtual list implementation
Implementation steps
-
Specify some variables
- Length = parseInt(scrollTop/itemHeight) Specifies the number of items to scroll up
- Height total height
- Total Total number of data items
- ItemHeight Height of a single piece of data
- The num window shows the total amount of data
- Start = length Indicates the start position for querying data
-
The container scroll bar needs an empty element to hold up, height = total * itemHeight
-
Actual window height = num * itemHeight
-
In this case, we just need to get the scrollTop in the Scroll event, so we know the length, and then update the actual marginTop(Length * itemHeight) of the dom, so that our content is still in the window
-
The above step ensures that the actual content container is in the window. Now we need to get the data that we want to display, i.e. from length to length + num, and then update it
Code:
< HTML > <head> <meta charset=" utF-8 "/> <meta name="viewport" content="width=device-width,initial-scale=1.0" /> <script SRC ="https://unpkg.com/vue@next"></script> <title> Virtual list </title> <style>. border: 1px solid; overflow-y: auto; height: var(--containerHeight); } .scroll-blank { height: var(--height); } .scroll { margin-top: var(--marginTop); } .scroll-item { height: var(--itemHeight); /* background-color: pink; */ } </style> </head> <body> <div id="app"> <div ref='container' class="container" :style="{ '--containerHeight': containerHeight + 'px', '--height': height + 'px', '--itemHeight': itemHeight + 'px', '--marginTop': marginTop + 'px', }" > <div class="scroll-blank"></div> <div class="scroll"> <div v-for='(item, index) in activeList' :key='item' class="scroll-item">{{ item }}</div> </div> </div> </div> <script> const { computed, onMounted, ref } = Vue const createList = () => { const list = [] let i = 0 while (i < 1000) { list.push(i++) } return list } const App = { setup() { const container = ref(null) const list = createList() const num = 20 const itemHeight = 20 const containerHeight = num * itemHeight const height = list.length * itemHeight let start = ref(0) let marginTop = ref(0) const activeList = computed(() => { const index = start.value return list.slice(index, index + 20) }) onMounted(() => { container.value.addEventListener('scroll', ({ target }) => { const { scrollTop } = target const itemNum = scrollTop / itemHeight start.value = parseInt(itemNum) marginTop.value = scrollTop }) }) return { container, marginTop, containerHeight, height, itemHeight, activeList } } }; const app = Vue.createApp(App); app.mount("#app"); </script> </body> </html>Copy the codeStorage and retrieval of 100,000 pieces of data
The above Vue3 simple implementation of virtual list, the rest of the data is how to store, retrieval. If all the data is in the main thread’s memory, it feels like a little bit of a split, and then webWorker and indexedDB come to mind!
webWorker
Start a thread outside the main thread to receive massive data without affecting the main thread
Reference documents:
- Developer.mozilla.org/zh-CN/docs/…
- www.ruanyifeng.com/blog/2018/0…
indexedDB
Can do mass data storage, and support retrieval
Reference documents:
- Developer.mozilla.org/zh-CN/docs/…
- www.ruanyifeng.com/blog/2018/0…
implementation
Set up service
First of all, for the convenience of preview, use KOA to build a service first, then also debugging
const Koa = require('koa') const Router = require('koa-router'); Const fs = require('fs/promises') const app = new Koa() const Router = new router () // implement route.get ('/', async (ctx, next) => { const content = await fs.readFile('./src/index.html', { encoding: 'utf8' }) ctx.body = content }) // worker router.get('/worker.js', async (ctx, next) => { const content = await fs.readFile('./src/worker.js') ctx.body = content }) App.use (router-.routes ()).use(router-.allowedmethods ()).listen(9527) console.log(' preview: `, `\x1B[36mhttp://localhost:9527\x1B[0m`)Copy the codeWorker part code
Const createDatabase = async () => {let version = 1 const databases = await indexedDB.databases() const preDatabase = databases.find(({ name }) => name === databaseName) if (preDatabase) version = preDatabase.version + 1 // indexedDB.deleteDatabase(databaseName) const database = indexedDB.open(databaseName, version) return new Promise((f, r) => { database.onsuccess = e => controlDatabase(e.target.result) database.onupgradeneeded = f database.onerror = r }) Const createTable = db => {return new Promise((f, r) => { db.deleteObjectStore('list') const table = db.createObjectStore('list', { keyPath: 'id' }) table.transaction.oncomplete = () => { const store = db.transaction('list', 'readwrite').objectStore('list') let i = 0 while (i < 100000) { store.add({ id: i++, num: Math.random()})} f(I) isFinished = true}})} // Retrieve const search = ({size, start}) => {if (! isFinished) return [] return new Promise((f, r) => { const store = db.transaction('list', 'readonly').objectStore('list') const range = IDBKeyRange.bound(start, start + size) const list = [] store.openCursor(range).onsuccess = ({ target: { result } }) => { if (! result) { return f(list) } list.push(result.value) result.continue() } }) }Copy the code