如何將項目添加到collection.Counter？然後將它們分類爲ASC？

Question

目前我正在通過計算600個文件（400封電子郵件和200封垃圾郵件）中的單詞出現次數來嘗試處理lingspam dataset。我已經通過Porter Stemmer Aglorithm使每個單詞具有通用性，我還希望我的結果能夠在每個文件中進行標準化處理以便進一步處理。但我我如何能做到這一點不確定..

資源迄今

• 8.3. collections — Container datatypes
• How to count co-ocurrences with collections.Counter() in python?
• Bag of Words model

爲了得到下面的輸出我需要能夠按升序添加文件內可能不存在的項目。

printing from ./../lingspam_results/spmsgb164.txt.out 
[('money', 0, 'univers', 0, 'sales', 0)] 
printing from ./../lingspam_results/spmsgb166.txt.out 
[('money', 2, 'univers', 0, 'sales', 0)] 
printing from ./../lingspam_results/spmsgb167.txt.out 
[('money', 0, 'univers', 0, 'sales', 1)] 

然後我打算轉換成vectors使用numpy。

[0,0,0] 
[2,0,0] 
[0,0,0] 

，而不是..

printing from ./../lingspam_results/spmsgb165.txt.out 
[] 
printing from ./../lingspam_results/spmsgb166.txt.out 
[('univers', 2)] 
printing from ./../lingspam_results/spmsgb167.txt.out 
[('sale', 1)] 

我怎樣才能規範從Counter模塊我的成果轉化爲Ascending Order（同時也增加項目的計數結果，可能無法從我的search_list存在）？我已經嘗試了下面的一些內容，只是從每個文本文件中讀取並根據search_list創建一個列表。

import numpy as np, os 
from collections import Counter 

def parse_bag(directory, search_list): 
    words = [] 
    for (dirpath, dirnames, filenames) in os.walk(directory): 
     for f in filenames: 
      path = directory + "/" + f 
      count_words(path, search_list) 
    return; 

def count_words(filename, search_list): 
    textwords = open(filename, 'r').read().split() 
    filteredwords = [t for t in textwords if t in search_list] 
    wordfreq = Counter(filteredwords).most_common(5) 
    print "printing from " + filename 
    print wordfreq 

search_list = ['sale', 'univers', 'money'] 
parse_bag("./../lingspam_results", search_list) 

感謝

Answer 1

從你的問題，這聽起來像你的要求是你想要t他在所有文件中以相同的順序排列相同的單詞，並計數。這應該爲你做它：

def count_words(filename, search_list): 
    textwords = open(filename, 'r').read().split() 
    filteredwords = [t for t in textwords if t in search_list] 
    counter = Counter(filteredwords) 
    for w in search_list: 
     counter[w] += 0  # ensure exists 
    wordfreq = sorted(counter.items()) 
    print "printing from " + filename 
    print wordfreq 

search_list = ['sale', 'univers', 'money'] 

輸出樣本：

printing from ./../lingspam_results/spmsgb164.txt.out 
[('money', 0), ('sale', 0), ('univers', 0)] 
printing from ./../lingspam_results/spmsgb166.txt.out 
[('money', 2), ('sale', 0), ('univers', 0)] 
printing from ./../lingspam_results/spmsgb167.txt.out 
[('money', 0), ('sale', 1), ('univers', 0)] 

要使用most_common可言，因爲你特別不希望每個文件的內容會影響我不認爲排序或列表長度。

Answer 2

呼叫Counter(filteredwords)，你在你的例子使用可以指望所有的單詞，就像你打算 - 它不就是給你最常用的 - 即存在沒有「most_common」方法 - 對於您必須重新處理纔能有元組contaning的（頻率字）的序列中的計數器的所有項目，和排序是：

def most_common(counter, n=5): 
    freq = sorted (((value ,item) for item, value in counter.viewitems()), reverse=True) 
    return [item[1] for item in freq[:n]] 

Answer 3

兩者結合jsbueno和沐心

def count_words_SO(filename, search_list): 
    textwords = open(filename, 'r').read().split() 
    filteredwords = [t for t in textwords if t in search_list] 
    counter = Counter(filteredwords) 
    for w in search_list: 
     counter[w] += 0  # ensure exists 
    wordfreq = number_parse(counter) 
    print "printing from " + filename 
    print wordfreq 

def number_parse(counter, n=5): 
    freq = sorted (((value ,item) for item, value in counter.viewitems()), reverse=True) 
    return [item[0] for item in freq[:n]] 

的隆重推出，只是更多的工作，我會立刻把它準備好了Neurel Network感謝所有:)

printing from ./../lingspam_results/spmsgb19.txt.out 
[0, 0, 0] 
printing from ./../lingspam_results/spmsgb2.txt.out 
[4, 0, 0] 
printing from ./../lingspam_results/spmsgb20.txt.out 
[10, 0, 0]