This article contains my series, Gitee open source address: gitee.com/mydb/interv…

List de-duplication refers to the process of removing duplicate elements from a List. This topic tests your ability to understand and use List iterators, Set collections, and new features in JDK 8.

There are three ways to implement List de-duplication:

  1. If there are more than one element, delete the duplicate item. Loop through the whole set and finally get a List with no duplicate elements.
  2. Use Set Set to de-weight, use Set Set itself de-weight function features, to achieve List de-weight;
  3. Use the Stream de-duplication feature in JDK 8.

1. Customize deduplication

There are two kinds of custom and the realization method, first of all, we can create a new collection, collection cycle through the original judgment cycle of elements, whether already exists in the new collection, if there is no insert, or ignored, finish this cycle, the resulting new collection is a set without repeating elements, specific implementation code is as follows:

import lombok.Data;

import java.util.ArrayList;
import java.util.List;

public class DistinctExample {
    public static void main(String[] args) {
        // Create and assign a value to List
        List<Person> list = new ArrayList<>();
        list.add(new Person("Bill"."123456".20));
        list.add(new Person("Zhang"."123456".18));
        list.add(new Person("Fifty"."123456".22));
        list.add(new Person("Zhang"."123456".18));
        // delete operation
        List<Person> newList = new ArrayList<>(list.size());
        list.forEach(i -> {
            if(! newList.contains(i)) {// Insert if the new collection does not existnewList.add(i); }});// Print the collectionnewList.forEach(p -> System.out.println(p)); }}@Data
class Person {
    private String name;
    private String password;
    private int age;

    public Person(String name, String password, int age) {
        this.name = name;
        this.password = password;
        this.age = age; }}Copy the code

The results of the above program are shown below:

The second method to implement the self-defined de-duplication function is to use the iterator to loop and determine whether the first occurrence position of the current element (indexOf) is equal to the last occurrence position (lastIndexOf). If not, it indicates that the element is a repeated element. Delete the current element, so that a set without repeating elements can be obtained after the loop. The implementation code is as follows:

import lombok.Data;

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;

public class DistinctExample {
    public static void main(String[] args) {
        // Create and assign a value to List
        List<Person> list = new ArrayList<>();
        list.add(new Person("Bill"."123456".20));
        list.add(new Person("Zhang"."123456".18));
        list.add(new Person("Fifty"."123456".22));
        list.add(new Person("Zhang"."123456".18));
        // delete operation
        Iterator<Person> iterator = list.iterator();
        while (iterator.hasNext()) {
            // Get the value of the loop
            Person item = iterator.next();
            // If there are two identical values
            if(list.indexOf(item) ! = list.lastIndexOf(item)) {// Remove the same valueiterator.remove(); }}// Prints the collection informationlist.forEach(p -> System.out.println(p)); }}@Data
class Person {
    private String name;
    private String password;
    private int age;

    public Person(String name, String password, int age) {
        this.name = name;
        this.password = password;
        this.age = age; }}Copy the code

The results of the above program are shown below:

2. Use Set to remove weights

A Set can be created by passing a List to the Set, so that it can implement data transfer and deduplication functions, specific implementation code is as follows:

import lombok.Data;

import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;

public class DistinctExample {
    public static void main(String[] args) {
        // Create and assign a value to List
        List<Person> list = new ArrayList<>();
        list.add(new Person("Bill"."123456".20));
        list.add(new Person("Zhang"."123456".18));
        list.add(new Person("Fifty"."123456".22));
        list.add(new Person("Zhang"."123456".18));
        // delete operation
        HashSet<Person> set = new HashSet<>(list);
        // Prints the collection informationset.forEach(p -> System.out.println(p)); }}@Data
class Person {
    private String name;
    private String password;
    private int age;

    public Person(String name, String password, int age) {
        this.name = name;
        this.password = password;
        this.age = age; }}Copy the code

The results of the above program are shown below:The problem with the above results is that the order of the elements changes after the HashSet is used. To solve this problem, we can use LinkedHashSet to implement de-duplication as follows:

import lombok.Data;

import java.util.ArrayList;
import java.util.LinkedHashSet;
import java.util.List;

public class DistinctExample {
    public static void main(String[] args) {
        // Create and assign a value to List
        List<Person> list = new ArrayList<>();
        list.add(new Person("Bill"."123456".20));
        list.add(new Person("Zhang"."123456".18));
        list.add(new Person("Fifty"."123456".22));
        list.add(new Person("Zhang"."123456".18));
        // delete operation
        LinkedHashSet<Person> set = new LinkedHashSet<>(list);
        // Prints the collection informationset.forEach(p -> System.out.println(p)); }}@Data
class Person {
    private String name;
    private String password;
    private int age;

    public Person(String name, String password, int age) {
        this.name = name;
        this.password = password;
        this.age = age; }}Copy the code

The results of the above program are shown below:

3. Use Stream for de-duplication

The last and simplest way to do this is to use a Stream provided in JDK 8 that contains a distinct method: Stream, which can directly implement the deduplication function of a collection. The code is as follows:

import lombok.Data;

import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;

public class DistinctExample {
    public static void main(String[] args) {
        // Create and assign a value to List
        List<Person> list = new ArrayList<>();
        list.add(new Person("Bill"."123456".20));
        list.add(new Person("Zhang"."123456".18));
        list.add(new Person("Fifty"."123456".22));
        list.add(new Person("Zhang"."123456".18));
        // delete operation
        list = list.stream().distinct().collect(Collectors.toList());
        // Prints the collection informationlist.forEach(p -> System.out.println(p)); }}@Data
class Person {
    private String name;
    private String password;
    private int age;

    public Person(String name, String password, int age) {
        this.name = name;
        this.password = password;
        this.age = age; }}Copy the code

The results of the above program are shown below:

conclusion

This paper introduces three ways to implement List de-duplication. Among them, the self-defined de-duplication function is relatively cumbersome to implement, while the Set Set can easily implement de-duplication function by relying on its own de-duplication feature. Moreover, the LinkedHashSet can be used to de-duplication while ensuring that the location of elements will not be changed. The last method, a distinct method in Stream, is new in JDK 8. It is the preferred method to implement the deduplication function because it is simple to write and does not require the creation of a new collection.

Judge right and wrong from yourself, praise to listen to others, gain and loss in the number.

Blogger introduction: programmer after 80, writing blog this matter “insist” for 12 years, hobbies: reading, jogging, badminton.

My public number: Java interview analysis