如何根據頻率對文本列進行分類

我有一個我希望根據頻率進行分類的城市名稱列表。我首先想要使用binning，但是因爲這需要單調的間距，所以我放棄了這一點。接下來，甚至更好的方法是使用pandas.qcut根據頻率創建基於分位數的類別。但擁有分位數，我不知道如何根據分位數創建一個額外的列。例如：如何根據頻率對文本列進行分類

import numpy as np 
import pandas as pd 

np.random.seed(0) 
cities = np.random.choice(['Ontario', 'Ottawa', 'Vancouver','Edmonton', 
          'Winnipeg', 'Churchill'], 500) 
# Create fake data and their frequencies 
df = pd.DataFrame (cities, columns=['City']) 
freq = df['City'].value_counts() 
print (freq) 
# Create quantiles 
qc = pd.qcut (freq, 3) 
print (qc) 
# And now? I have the quantiles but how to assign a categorie to each City? 
category_for_each_city = df['City'] in qC# does not work, but many other things neither

我嘗試了很多事情，但都沒有成功。我應該能夠爲此編寫一個循環，但我無法想象這是Python的方式。我試圖尋找一些sklearn變形金剛，但無法找到任何與此特定的解決方案。任何幫助將不勝感激。

此外，我有很多傾斜的分佈，可以擴展到例如日誌轉換的解決方案將有很大的幫助。

來源

2017-10-08 Arnold

你幾乎沒有...

In [106]: category_for_each_city = df['City'].map(qc) 

In [107]: category_for_each_city 
Out[107]: 
0  (77.333, 84.667] 
1  (72.999, 77.333] 
2  (84.667, 100.0] 
3  (84.667, 100.0] 
4  (84.667, 100.0] 
5  (84.667, 100.0] 
6  (77.333, 84.667] 
      ... 
493  (84.667, 100.0] 
494 (72.999, 77.333] 
495 (77.333, 84.667] 
496  (84.667, 100.0] 
497 (77.333, 84.667] 
498 (77.333, 84.667] 
499 (77.333, 84.667] 
Name: City, Length: 500, dtype: category 
Categories (3, interval[float64]): [(72.999, 77.333] < (77.333, 84.667] < (84.667, 100.0]]

UPDATE：

In [114]: qc = pd.qcut (freq, 3, labels=[0,1,2]) 

In [115]: category_for_each_city = df['City'].map(qc) 

In [116]: category_for_each_city 
Out[116]: 
0  1 
1  0 
2  2 
3  2 
4  2 
5  2 
6  1 
     .. 
493 2 
494 0 
495 1 
496 2 
497 1 
498 1 
499 1 
Name: City, Length: 500, dtype: category 
Categories (3, int64): [0 < 1 < 2]

來源

2017-10-08 10:25:52 MaxU

這很容易的確。我正在嘗試類似'qc.category_for_each_city.codes [df ['City']]'，但您的解決方案要簡單得多。非常感謝你的幫助！ – Arnold

如何根據頻率對文本列進行分類

回答

相關問題