將Pandas DataFrame轉換爲橙色表

Question

我注意到這是一個issue on GitHub already。有沒有人有任何代碼將熊貓數據框轉換爲橙色表格？

明確地說，我有下表。

 user hotel star_rating user home_continent gender 
0   1  39   4.0  1    2 female 
1   1  44   3.0  1    2 female 
2   2  63   4.5  2    3 female 
3   2  2   2.0  2    3 female 
4   3  26   4.0  3    1 male 
5   3  37   5.0  3    1 male 
6   3  63   4.5  3    1 male 

Answer 1

Orange軟件包的文檔沒有涵蓋所有細節。根據lib_kernel.cpp，Table._init__(Domain, numpy.ndarray)僅適用於int和float。

他們確實應該爲pandas.DataFrames或至少支持numpy.dtype("str")提供一個C級接口。

更新：添加table2df,df2table通過對int和float使用numpy大大提高了性能。

將這段腳本保存在您的橙色Python腳本集合中，現在您在橙色環境中配備了熊貓。

使用：a_pandas_dataframe = table2df(a_orange_table)，a_orange_table = df2table(a_pandas_dataframe)

注意：此腳本只能在Python 2.x中，參考@DustinTang的answer爲Python 3.x的兼容腳本。

import pandas as pd 
import numpy as np 
import Orange 

#### For those who are familiar with pandas 
#### Correspondence: 
#### value <-> Orange.data.Value 
####  NaN <-> ["?", "~", "."] # Don't know, Don't care, Other 
#### dtype <-> Orange.feature.Descriptor 
####  category, int <-> Orange.feature.Discrete # category: > pandas 0.15 
####  int, float <-> Orange.feature.Continuous # Continuous = core.FloatVariable 
####             # refer to feature/__init__.py 
####  str <-> Orange.feature.String 
####  object <-> Orange.feature.Python 
#### DataFrame.dtypes <-> Orange.data.Domain 
#### DataFrame.DataFrame <-> Orange.data.Table = Orange.orange.ExampleTable 
####        # You will need this if you are reading sources 

def series2descriptor(d, discrete=False): 
    if d.dtype is np.dtype("float"): 
     return Orange.feature.Continuous(str(d.name)) 
    elif d.dtype is np.dtype("int"): 
     return Orange.feature.Continuous(str(d.name), number_of_decimals=0) 
    else: 
     t = d.unique() 
     if discrete or len(t) < len(d)/2: 
      t.sort() 
      return Orange.feature.Discrete(str(d.name), values=list(t.astype("str"))) 
     else: 
      return Orange.feature.String(str(d.name)) 


def df2domain(df): 
    featurelist = [series2descriptor(df.icol(col)) for col in xrange(len(df.columns))] 
    return Orange.data.Domain(featurelist) 


def df2table(df): 
    # It seems they are using native python object/lists internally for Orange.data types (?) 
    # And I didn't find a constructor suitable for pandas.DataFrame since it may carry 
    # multiple dtypes 
    # --> the best approximate is Orange.data.Table.__init__(domain, numpy.ndarray), 
    # --> but the dtype of numpy array can only be "int" and "float" 
    # --> * refer to src/orange/lib_kernel.cpp 3059: 
    # --> * if (((*vi)->varType != TValue::INTVAR) && ((*vi)->varType != TValue::FLOATVAR)) 
    # --> Documents never mentioned >_< 
    # So we use numpy constructor for those int/float columns, python list constructor for other 

    tdomain = df2domain(df) 
    ttables = [series2table(df.icol(i), tdomain[i]) for i in xrange(len(df.columns))] 
    return Orange.data.Table(ttables) 

    # For performance concerns, here are my results 
    # dtndarray = np.random.rand(100000, 100) 
    # dtlist = list(dtndarray) 
    # tdomain = Orange.data.Domain([Orange.feature.Continuous("var" + str(i)) for i in xrange(100)]) 
    # tinsts = [Orange.data.Instance(tdomain, list(dtlist[i]))for i in xrange(len(dtlist))] 
    # t = Orange.data.Table(tdomain, tinsts) 
    # 
    # timeit list(dtndarray) # 45.6ms 
    # timeit [Orange.data.Instance(tdomain, list(dtlist[i])) for i in xrange(len(dtlist))] # 3.28s 
    # timeit Orange.data.Table(tdomain, tinsts) # 280ms 

    # timeit Orange.data.Table(tdomain, dtndarray) # 380ms 
    # 
    # As illustrated above, utilizing constructor with ndarray can greatly improve performance 
    # So one may conceive better converter based on these results 


def series2table(series, variable): 
    if series.dtype is np.dtype("int") or series.dtype is np.dtype("float"): 
     # Use numpy 
     # Table._init__(Domain, numpy.ndarray) 
     return Orange.data.Table(Orange.data.Domain(variable), series.values[:, np.newaxis]) 
    else: 
     # Build instance list 
     # Table.__init__(Domain, list_of_instances) 
     tdomain = Orange.data.Domain(variable) 
     tinsts = [Orange.data.Instance(tdomain, [i]) for i in series] 
     return Orange.data.Table(tdomain, tinsts) 
     # 5x performance 


def column2df(col): 
    if type(col.domain[0]) is Orange.feature.Continuous: 
     return (col.domain[0].name, pd.Series(col.to_numpy()[0].flatten())) 
    else: 
     tmp = pd.Series(np.array(list(col)).flatten()) # type(tmp) -> np.array(dtype=list (Orange.data.Value)) 
     tmp = tmp.apply(lambda x: str(x[0])) 
     return (col.domain[0].name, tmp) 

def table2df(tab): 
    # Orange.data.Table().to_numpy() cannot handle strings 
    # So we must build the array column by column, 
    # When it comes to strings, python list is used 
    series = [column2df(tab.select(i)) for i in xrange(len(tab.domain))] 
    series_name = [i[0] for i in series] # To keep the order of variables unchanged 
    series_data = dict(series) 
    print series_data 
    return pd.DataFrame(series_data, columns=series_name) 

Answer 2

像這樣？

table = Orange.data.Table(df.as_matrix()) 

Orange中的列將獲得通用名稱（a1，a2 ...）。如果要從數據框中複製名稱和類型，請從數據框中構建Orange.data.Domain對象（http://docs.orange.biolab.si/reference/rst/Orange.data.domain.html#Orange.data.Domain.init），並將其作爲上面的第一個參數傳遞。

請參閱http://docs.orange.biolab.si/reference/rst/Orange.data.table.html中的構造函數。

Answer 3

爲了將pandas DataFrame轉換爲橙色表，您需要構建一個指定列類型的域。

對於連續變量，您只需提供變量的名稱，但對於離散變量，還需要提供所有可能值的列表。

下面的代碼將構造一個域名爲您的數據幀，並將其轉換爲橙色表：

import numpy as np 
from Orange.feature import Discrete, Continuous 
from Orange.data import Domain, Table 
domain = Domain([ 
    Discrete('user', values=[str(v) for v in np.unique(df.user)]), 
    Discrete('hotel', values=[str(v) for v in np.unique(df.hotel)]), 
    Continuous('star_rating'), 
    Discrete('user', values=[str(v) for v in np.unique(df.user)]), 
    Discrete('home_continent', values=[str(v) for v in np.unique(df.home_continent)]), 
    Discrete('gender', values=['male', 'female'])], False) 
table = Table(domain, [map(str, row) for row in df.as_matrix()]) 

地圖（STR，行）所需步驟，橙色知道，數據中包含的離散特徵值（而不是值列表中的值的索引）。

Answer 4

此代碼從@TurtleIzzy修改爲Python3。

import numpy as np 
from Orange.data import Table, Domain, ContinuousVariable, DiscreteVariable 


def series2descriptor(d): 
    if d.dtype is np.dtype("float") or d.dtype is np.dtype("int"): 
     return ContinuousVariable(str(d.name)) 
    else: 
     t = d.unique() 
     t.sort() 
     return DiscreteVariable(str(d.name), list(t.astype("str"))) 

def df2domain(df): 
    featurelist = [series2descriptor(df.iloc[:,col]) for col in range(len(df.columns))] 
    return Domain(featurelist) 

def df2table(df): 
    tdomain = df2domain(df) 
    ttables = [series2table(df.iloc[:,i], tdomain[i]) for i in range(len(df.columns))] 
    ttables = np.array(ttables).reshape((len(df.columns),-1)).transpose() 
    return Table(tdomain , ttables) 

def series2table(series, variable): 
    if series.dtype is np.dtype("int") or series.dtype is np.dtype("float"): 
     series = series.values[:, np.newaxis] 
     return Table(series) 
    else: 
     series = series.astype('category').cat.codes.reshape((-1,1)) 
     return Table(series)