導入CSV與兩個分隔符

Question

我有兩個分隔符（;）和CSV大熊貓（,）它看起來像這樣：

vin;vorgangid;eventkm;D_8_lamsoni_w_time;D_8_lamsoni_w_value 
V345578;295234545;13;-1000.0,-980.0;7.9921875,11.984375 
V346670;329781064;13;-960.0,-940.0;7.9921875,11.984375 

我想將它導入到大熊貓的數據幀，與（ ;）作爲列分隔符和（,）作爲list或array的分隔符，使用float作爲數據類型。到目前爲止，我正在使用這種方法，但我確信有更容易的東西。使用;作爲分隔符

aa=0; 
csv_import=pd.read_csv(folder+FileName, ';') 
for col in csv_import.columns: 
aa=aa+1 
if type(csv_import[col][0])== str and aa>3: 
    # string to list of strings 
    csv_import[col]=csv_import[col].apply(lambda x:x.split(',')) 
    # make the list of stings into a list of floats 
    csv_import[col]=csv_import[col].apply(lambda x: [float(y) for y in x]) 

Answer 1

旁白fr在這裏的其他很好的答案，這是更多的熊貓特定的，應該指出的是，Python本身在字符串處理方面非常強大。你可以只放在一個StringIO對象與','更換​​3210的結果，並從那裏正常工作：

In [8]: import pandas as pd 

In [9]: from cStringIO import StringIO 

In [10]: pd.read_csv(StringIO(''.join(l.replace(';', ',') for l in open('stuff.csv')))) 
Out[10]: 
        vin vorgangid eventkm D_8_lamsoni_w_time \ 
V345578 295234545 13 -1000.0 -980.0   7.992188 
V346670 329781064 13  -960.0 -940.0   7.992188 

        D_8_lamsoni_w_value 
V345578 295234545   11.984375 
V346670 329781064   11.984375 

Answer 2

第一讀取CSV：

df = pd.read_csv(filename, sep=';') 

UPDATE：

In [67]: num_cols = df.columns.difference(['vin','vorgangid','eventkm']) 

In [68]: num_cols 
Out[68]: Index(['D_8_lamsoni_w_time', 'D_8_lamsoni_w_value'], dtype='object') 

In [69]: df[num_cols] = (df[num_cols].apply(lambda x: x.str.split(',', expand=True) 
    ....:            .stack() 
    ....:            .astype(float) 
    ....:            .unstack() 
    ....:            .values.tolist()) 
    ....:    ) 

In [70]: df 
Out[70]: 
     vin vorgangid eventkm D_8_lamsoni_w_time  D_8_lamsoni_w_value 
0 V345578 295234545  13 [-1000.0, -980.0] [7.9921875, 11.984375] 
1 V346670 329781064  13 [-960.0, -940.0] [7.9921875, 11.984375] 

In [71]: type(df.loc[0, 'D_8_lamsoni_w_value'][0]) 
Out[71]: float 

OLD答案：

現在，我們可以數分成列表S IN的 「數量」 列：

In [20]: df[['D_8_lamsoni_w_time', 'D_8_lamsoni_w_value']] = \ 
    df[['D_8_lamsoni_w_time', 'D_8_lamsoni_w_value']].apply(lambda x: x.str.split(',')) 
In [21]: df 
Out[21]: 
     vin vorgangid eventkm D_8_lamsoni_w_time  D_8_lamsoni_w_value 
0 V345578 295234545  13 [-1000.0, -980.0] [7.9921875, 11.984375] 
1 V346670 329781064  13 [-960.0, -940.0] [7.9921875, 11.984375] 

Answer 3

可以在read_csv使用參數converters和定義自定義功能的分裂：

def f(x): 
    return [float(i) for i in x.split(',')] 

#after testing replace io.StringIO(temp) to filename 
df = pd.read_csv(io.StringIO(temp), 
       sep=";", 
       converters={'D_8_lamsoni_w_time':f, 'D_8_lamsoni_w_value':f}) 
print (df) 
     vin vorgangid eventkm D_8_lamsoni_w_time  D_8_lamsoni_w_value 
0 V345578 295234545  13 [-1000.0, -980.0] [7.9921875, 11.984375] 
1 V346670 329781064  13 [-960.0, -940.0] [7.9921875, 11.984375] 

另一種解決方案與NaN在4.和5.列工作：

您可以使用read_csv與分隔符;，然後將str.split應用於4.和5.列由iloc選擇和list到float轉換每個值：

import pandas as pd 
import numpy as np 
import io 

temp=u"""vin;vorgangid;eventkm;D_8_lamsoni_w_time;D_8_lamsoni_w_value 
V345578;295234545;13;-1000.0,-980.0;7.9921875,11.984375 
V346670;329781064;13;-960.0,-940.0;7.9921875,11.984375""" 
#after testing replace io.StringIO(temp) to filename 
df = pd.read_csv(io.StringIO(temp), sep=";") 

print (df) 
     vin vorgangid eventkm D_8_lamsoni_w_time D_8_lamsoni_w_value 
0 V345578 295234545  13  -1000.0,-980.0 7.9921875,11.984375 
1 V346670 329781064  13  -960.0,-940.0 7.9921875,11.984375 

#split 4.th and 5th column and convert to numpy array 
df.iloc[:,3] = df.iloc[:,3].str.split(',').apply(lambda x: [float(i) for i in x]) 
df.iloc[:,4] = df.iloc[:,4].str.split(',').apply(lambda x: [float(i) for i in x]) 
print (df) 
     vin vorgangid eventkm D_8_lamsoni_w_time  D_8_lamsoni_w_value 
0 V345578 295234545  13 [-1000.0, -980.0] [7.9921875, 11.984375] 
1 V346670 329781064  13 [-960.0, -940.0] [7.9921875, 11.984375] 

如果需要numpy arrays代替lists：

#split 4.th and 5th column and convert to numpy array 
df.iloc[:,3] = df.iloc[:,3].str.split(',').apply(lambda x: np.array([float(i) for i in x])) 
df.iloc[:,4] = df.iloc[:,4].str.split(',').apply(lambda x: np.array([float(i) for i in x])) 
print (df) 
     vin vorgangid eventkm D_8_lamsoni_w_time  D_8_lamsoni_w_value 
0 V345578 295234545  13 [-1000.0, -980.0] [7.9921875, 11.984375] 
1 V346670 329781064  13 [-960.0, -940.0] [7.9921875, 11.984375] 

print (type(df.iloc[0,3])) 
<class 'numpy.ndarray'> 

---

我儘量提高你的solutiuon：

a=0; 
csv_import=pd.read_csv(folder+FileName, ';') 
for col in csv_import.columns: 
    a += 1 
    if type(csv_import.ix[0, col])== str and a>3: 
     # string to list of strings 
     csv_import[col]=csv_import[col].apply(lambda x: [float(y) for y in x.split(',')])