如何將帶有鋸齒陣列的字典列表轉換爲數據框

Question

對熊貓來說是新的。我試圖讓一個JSON數據集，看起來像這樣

data = [ 
    {'id':1, 'colors':['red', 'blue']}, 
    {'id':2, 'colors':['red', 'blue', 'green']}, 
    {'id':3, 'colors':['orange', 'blue', 'orange']}, 
] 

成熊貓數據幀，看起來像這樣

import pandas as pd 
df = pd.DataFrame({'id':[1,2,3], 
        'blue':[1,1,1], 
        'green':[0,1,0], 
        'orange':[0,0,2], 
        'red':[1,1,0]}) 
df 
    blue green id orange red 
0  1  0 1  0 1 
1  1  1 2  0 1 
2  1  0 3  2 0 

其中列有「身份證」和獨特的顏色和行是每個原始字典中每種顏色的ID和計數。我該怎麼做呢？

Answer 1

想通了。

df = [DataFrame(e) for e in data] 
df = pd.concat(df) 
df = df.pivot_table(index=['id'], columns=['colors'], aggfunc=len).fillna(0) 
df 

    blue green id orange red 
0  1  0 1  0 1 
1  1  1 2  0 1 
2  1  0 3  2 0 

Answer 2

>>> ids = [item['id'] for item in data] 
>>> col = [item['colors'] for item in data] 

>>> ids = np.repeat(ids, list(map(len, col))) 
>>> col = [a for item in col for a in item] 

>>> df = DataFrame({'ids':ids, 'colors':col}) 
>>> df 
    colors id 
0  red 1 
1 blue 1 
2  red 2 
3 blue 2 
4 green 2 
5 orange 3 
6 blue 3 
7 orange 3 
>>> df.groupby(['id', 'colors']).size().unstack().fillna(0) 
colors blue green orange red 
id        
1   1  0  0 1 
2   1  1  0 1 
3   1  0  2 0 

你可以在結束通話.reset_index，有id爲一列，而不是指數。

Answer 3

我將首先把對colors列出使用set().union()所有獨特的顏色列表，然後創建一個使用collections.defaultdict新的字典。示例 -

In [10]: data = [ 
    ....:  {'id':1, 'colors':['red', 'blue']}, 
    ....:  {'id':2, 'colors':['red', 'blue', 'green']}, 
    ....:  {'id':3, 'colors':['orange', 'blue', 'orange']}, 
    ....: ] 

In [11]: 

In [11]: from collections import defaultdict 

In [24]: colorlist = list(set().union(*colorlist)) 

In [25]: colorlist 
Out[25]: ['orange', 'blue', 'red', 'green'] 

In [26]: newd = defaultdict(list) 

In [30]: for x in data: 
    ....:  xid = x['id'] 
    ....:  newd['id'].append(xid) 
    ....:  for elem in colorlist: 
    ....:   if elem in x['colors']: 
    ....:    newd[elem].append(xid) 
    ....:   else: 
    ....:    newd[elem].append(0) 
    ....: 

In [31]: newd 
Out[31]: defaultdict(<class 'list'>, {'id': [1, 2, 3], 'orange': [0, 0, 3], 'blue': [1, 2, 3], 'red': [1, 2, 0], 'green': [0, 2, 0]}) 

In [32]: df = pd.DataFrame(newd) 

In [33]: df 
Out[33]: 
    blue green id orange red 
0  1  0 1  0 1 
1  2  2 2  0 2 
2  3  0 3  3 0 

Answer 4

方法是用顏色計數，轉置和連接爲每一行創建一個DataFrame。可能有更好的方法可以避免爲每行創建熊貓物件造成的開銷。

import pandas as pd 

df = pd.io.json.json_normalize(
    [{'id':1, 'colors':['red', 'blue']}, 
    {'id':2, 'colors':['red', 'blue', 'green']}, 
    {'id':3, 'colors':['orange', 'blue', 'orange']},] 
    ).set_index('id') 

df = pd.concat(df.colors.map(lambda color_list: \ 
    pd.DataFrame(pd.Series(color_list).value_counts()).transpose())\ 
    .values).fillna(0).set_index(df.index) 

Answer 5

您可以使用列表理解和類型的字典中，對於一個更快的替代方案

In [2494]: pd.DataFrame(dict(zip(r['colors']+['id'], [1]*len(r['colors'])+[r['id']])) 
         for r in data).fillna(0) 
Out[2494]: 
    blue green id orange red 
0  1 0.0 1  0.0 1.0 
1  1 1.0 2  0.0 1.0 
2  1 0.0 3  1.0 0.0 

---

時序

In [2492]: %timeit pd.DataFrame(dict(zip(r['colors']+['id'], [1]*len(r['colors'])+[r['id']])) for r in data).fillna(0) 
1000 loops, best of 3: 863 µs per loop 

In [2493]: %timeit pd.concat([pd.DataFrame(e) for e in data]).pivot_table(index=['id'], columns=['colors'], aggfunc=len).fillna(0) 
100 loops, best of 3: 7.79 ms per loop 

---

詳細

In [2495]: [dict(zip(r['colors']+['id'], [1]*len(r['colors'])+[r['id']])) for r in data] 
Out[2495]: 
[{'blue': 1, 'id': 1, 'red': 1}, 
{'blue': 1, 'green': 1, 'id': 2, 'red': 1}, 
{'blue': 1, 'id': 3, 'orange': 1}]