preface

Deduplication is often encountered in the development of a hot issue, but the current project is encountered in the background interface using SQL deduplication, simple and efficient, basic will not let the front-end processing deduplication.

So what happens when the front-end deweights? If you display 10 different pieces of data per page, you may need to send multiple HTTP requests to filter out 10 different pieces of data to display 10 pieces of data. If you delete the data in the background, you can obtain 10 different pieces of data in a single HTTP request.

Of course, this is not to say that the front end to heavy is not necessary, still need to be skilled use. This article mainly introduces several common methods of array de-duplication.

Method implementation

Double cycle to heavy

For (or while) with a double loop is a clumsy way, its implementation principle is simple: first define an array containing the first element on the original array, and then traverse the original array, each element from the original array and compare the new each element of the array, if you don’t repeat is added to the new array, finally return to the new array; Because it’s O(n^2) time, if the array is large, it’s very memory intensive

function unique(arr) {
    if (!Array.isArray(arr)) {
        console.log('type error! ')
        return
    }
    let res = [arr[0]]
    for (let i = 1; i < arr.length; i++) {
        let flag = true
        for (let j = 0; j < res.length; j++) {
            if (arr[i] === res[j]) {
                flag = false;
                break}}if (flag) {
            res.push(arr[i])
        }
    }
    return res
}
Copy the code

The indexOf method deduplicates 1

The indexOf() method of an array returns the first occurrence of a specified element in the array. This method first defines an empty array res, then calls the indexOf method to iterate over the original array. If the element is not in the RES, it is pushed into the res, and the res is returned to obtain the deduplicated array

function unique(arr) {
    if (!Array.isArray(arr)) {
        console.log('type error! ')
        return
    }
    let res = []
    for (let i = 0; i < arr.length; i++) {
        if (res.indexOf(arr[i]) === - 1) {
            res.push(arr[i])
        }
    }
    return res
}
Copy the code

The indexOf method is deduplicated 2

Use indexOf to check whether the first occurrence of an element in the array is the same as the element’s current position. If not, the element is a duplicate element

function unique(arr) {
    if (!Array.isArray(arr)) {
        console.log('type error! ')
        return
    }
    return Array.prototype.filter.call(arr, function(item, index){
        return arr.indexOf(item) === index;
    });
}
Copy the code

Remove the weight of adjacent elements

This method first calls the array sort method sort(), and then according to the sorted results of traversal and adjacent elements comparison, if they are equal to skip the element changes, until the traversal is complete

function unique(arr) {
    if (!Array.isArray(arr)) {
        console.log('type error! ')
        return
    }
    arr = arr.sort()
    let res = []
    for (let i = 0; i < arr.length; i++) {
        if(arr[i] ! == arr[i- 1]) {
            res.push(arr[i])
        }
    }
    return res
}
Copy the code

Use object properties to deduplicate

Create an empty object, walk through the array, set the values in the array to the property of the object, and assign the initial value of 1 to the property. Each occurrence increases the value of the corresponding property by 1, so that the value of the property corresponds to the number of occurrences of the element

function unique(arr) {
    if (!Array.isArray(arr)) {
        console.log('type error! ')
        return
    }
    let res = [],
        obj = {}
    for (let i = 0; i < arr.length; i++) {
        if(! obj[arr[i]]) { res.push(arr[i]) obj[arr[i]] =1
        } else {
            obj[arr[i]]++
        }
    }
    return res
}
Copy the code

Set and deconstruct assignment to de-duplicate

ES6 has added the new data type set. One of the biggest features of set is that the data is not repeated. The Set function can take an array (or an array-like object) as an argument to initialize, and can be de-duplicated using this feature

function unique(arr) {
    if (!Array.isArray(arr)) {
        console.log('type error! ')
        return
    }
    return [...new Set(arr)]
}
Copy the code

Array.from and set are deduplicated

The array. from method converts the Set structure into an Array result, and we know that the Set result is a unique Set of data, so it can be deduplicated

function unique(arr) {
    if (!Array.isArray(arr)) {
        console.log('type error! ')
        return
    }
    return Array.from(new Set(arr))
}
Copy the code

conclusion

Array deduplication is a hot issue in development. We can choose different implementations according to different application scenarios.