Python unweights a key of the dictionary in the list
- I. Requirements description
- Second, my solution
- Write in the last
I. Requirements description
When we write crawlers, we often encounter data in JSON format, which usually has the following structure:
data = [{'name':'small K'.'score':100},
{'name':'the little J'.'score':98},
{'name':'the little Q'.'score':95},
{'name':'small K'.'score':100}]
Copy the code
It is obvious that the data with the name of small K is duplicated, so we need to carry out de-duplication. In general, we can use set() to deduplicate a list, that is:
data = list(set(data))
Copy the code
However, after you run it, you will find that it reported an error:
The data in the list can’t be dict, so what do you do?
Second, my solution
Define a function to filter the data according to a key:
def DelRepeat(data,key) :
new_data = [] Used to store the list after the rehash
values = [] # Used to store existing values
for d in data:
if d[key] not in values:
new_data.append(d)
values.append(d[key])
return new_data
Copy the code
Parameter data is the list to be removed, and key is the key to be removed. The result after the deletion is:
[{'name': 'small K'.'score': 100},
{'name': 'the little J'.'score': 98},
{'name': 'the little Q'.'score': 95}]
Copy the code
Success to heavy!
Write in the last
Finally, thank you for reading ~ with great patience
Creation is not easy, Sir, please stay... Have a cameo appearance with your cute hands (◕ܫ← danjun)