Python unweights a key of the dictionary in the list

  • I. Requirements description
  • Second, my solution
  • Write in the last

I. Requirements description

When we write crawlers, we often encounter data in JSON format, which usually has the following structure:

data = [{'name':'small K'.'score':100},
        {'name':'the little J'.'score':98},
        {'name':'the little Q'.'score':95},
        {'name':'small K'.'score':100}]
Copy the code

It is obvious that the data with the name of small K is duplicated, so we need to carry out de-duplication. In general, we can use set() to deduplicate a list, that is:

data = list(set(data))
Copy the code

However, after you run it, you will find that it reported an error:



The data in the list can’t be dict, so what do you do?

Second, my solution

Define a function to filter the data according to a key:

def DelRepeat(data,key) :
    new_data = [] Used to store the list after the rehash
    values = []   # Used to store existing values
    for d in data:
        if d[key] not in values:
            new_data.append(d)
            values.append(d[key])
    return new_data
Copy the code

Parameter data is the list to be removed, and key is the key to be removed. The result after the deletion is:

[{'name': 'small K'.'score': 100},
 {'name': 'the little J'.'score': 98},
 {'name': 'the little Q'.'score': 95}]
Copy the code

Success to heavy!

Write in the last

Finally, thank you for reading ~ with great patience

Creation is not easy, Sir, please stay... Have a cameo appearance with your cute hands (◕ܫ← danjun)