how to delete duplicate dictionaries in list of dictionaries and extract the values of dictionary?

Question

delete duplicate dictionaries in list of dictionary and extract the values of dictionary

id=842
mg=[ 
       {"ss": 81,  "rr": 842, "cd": 81},
            {"ss": 842, "rr": 81,  "cd": 81},
            {"ss": 81,  "rr": 842, "cd": 81},
            {"ss": 842,  "rr": 81, "cd": 81},
            {"ss": 82,  "rr": 842, "cd": 82},
             {"ss": 82,  "rr": 842, "cd": 82},
            {"ss": 83,  "rr": 842, "cd": 83},
            {"ss": 842,  "rr": 83, "cd": 83},
    ]
new_d = []
for x in mg:
    if x not in new_d:
        new_d.append(x) 
mg=new_d

mglist=[]   
length=len(mg)      
for i in range(length):
    if mg[i].rr==id:
        if mg[i] not in mglist:
            cr+=1
            mglist.append(mg[i])
    if mg[i].ss==id:
        if mg[i] not in mglist:
            cs+=1
            mglist.append(mg[i])
print(cr)
print(cs)

I want to delete duplicate rows and then count frequency of id (in first and second column) in this list of dictionary how I could solve my problem with iteration and remove duplication how to delete duplicate dictionaries in list of dictionaries and extract the values of dictionary?

This is not about data science, you could ask on https://stackoverflow.com/ — Erwan, Oct 13 '22 at 11:03

flyakite · Answer 1 · 2022-10-14T10:00:11.523

#remove duplicates
new_d = [i for n, i in enumerate(mg) if i not in mg[n + 1:]]

#check the data
cs_d = [i for n, i in enumerate(new_d) if i['ss'] == id]
cr_d = [i for n, i in enumerate(new_d) if i['rr'] == id]

#sample return
print(cs_d) #print dictionarys, where ss=id
print(len(cs_d)) #print number of elements, where ss=id

Or everything inside a single loop to increase performance:

cs_d = []
cr_d = []
for n, i in enumerate(mg):
    #check, if dict is unique
    if i not in mg[n + 1:]:
        #store cs
        if i['ss'] == id:
            cs_d.append(i)

        #store cr
        if i['rr'] == id:
            cr_d.append(i)

Alternatively, you could use a pandas.Dataframe:

import pandas as pd

df = pd.DataFrame(mg)

#get unique mg list
df_unique = df.drop_duplicates()

#count cs, where ss==id
cs = df_unique.ss.value_counts()[id]

#count cr, where rr==id
cr = df_unique.rr.value_counts()[id]

#extract values, where ss==id
print( df_unique.loc[df_unique.ss == id] )
#    ss  rr  cd
#1  842  81  81
#7  842  83  83

#extract values, where rr==id
print( df_unique.loc[df_unique.rr == id] )
#   ss   rr  cd
#0  81  842  81
#4  82  842  82
#6  83  842  83

thanks but how we can solve it, if we would like to do it without using Pandas? — user10296606, Oct 13 '22 at 20:47

how to delete duplicate dictionaries in list of dictionaries and extract the values of dictionary?

1 Answers1