如何使用panda.read_sas以塊形式讀取二進制壓縮的SAS文件並將其保存爲羽毛

Question

我試圖使用pandas.read_sas()以區塊形式讀取二進制壓縮的SAS文件，並將每個塊保存爲單獨的羽化文件。

這是我的代碼

import feather as fr 
import pandas as pd 

pdi = pd.read_sas("C:/data/test.sas7bdat", chunksize = 100000, iterator = True) 

i = 1 
for pdj in pdi: 
    fr.write_dataframe(pdj, 'C:/data/test' + str(i) + '.feather') 
    i = i + 1 

不過，我得到以下錯誤

ValueError Traceback (most recent call last) in() 1 i = 1 2 for pdj in pdi: ----> 3 fr.write_dataframe(pdj, 'C:/test' + str(i) + '.feather') 4 i = i + 1 5

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pyarrow\feather.py in write_feather(df, dest) 116 writer = FeatherWriter(dest) 117 try: --> 118 writer.write(df) 119 except: 120 # Try to make sure the resource is closed

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pyarrow\feather.py in write(self, df) 94 95 elif inferred_type not in ['unicode', 'string']: ---> 96 raise ValueError(msg) 97 98 if not isinstance(name, six.string_types):

ValueError: cannot serialize column 0 named SOME_ID with dtype bytes

我使用Windows 7和Python 3.6。當我最檢查它時，列的單元格被包裝在b'cell_value'中，我假定這些列是二進制格式的。

我是一個完整的Python初學者，所以不明白是什麼問題？

Answer 1

編輯：看起來這是在最近的版本修補了一個錯誤： https://issues.apache.org/jira/browse/ARROW-1672 https://github.com/apache/arrow/commit/238881fae8530a1ae994eb0e283e4783d3dd2855

是列名字符串？你確定pdj的類型是pd.DataFrame嗎？

Limitations

Some features of pandas are not supported in Feather:

Non-string column names

Row indexes

Object-type columns with non-homogeneous data