CountVectorizer：沒有安裝詞彙表

我通過傳遞vocabulary參數的詞彙來實例化sklearn.feature_extraction.text.CountVectorizer對象，但我得到一條sklearn.utils.validation.NotFittedError: CountVectorizer - Vocabulary wasn't fitted.錯誤消息。爲什麼？CountVectorizer：沒有安裝詞彙表

例子：

import sklearn.feature_extraction 
import numpy as np 
import pickle 

# Save the vocabulary 
ngram_size = 1 
dictionary_filepath = 'my_unigram_dictionary' 
vectorizer = sklearn.feature_extraction.text.CountVectorizer(ngram_range=(ngram_size,ngram_size), min_df=1) 

corpus = ['This is the first document.', 
     'This is the second second document.', 
     'And the third one.', 
     'Is this the first document? This is right.',] 

vect = vectorizer.fit(corpus) 
print('vect.get_feature_names(): {0}'.format(vect.get_feature_names())) 
pickle.dump(vect.vocabulary_, open(dictionary_filepath, 'w')) 

# Load the vocabulary 
vocabulary_to_load = pickle.load(open(dictionary_filepath, 'r')) 
loaded_vectorizer = sklearn.feature_extraction.text.CountVectorizer(ngram_range=(ngram_size,ngram_size), min_df=1, vocabulary=vocabulary_to_load) 
print('loaded_vectorizer.get_feature_names(): {0}'.format(loaded_vectorizer.get_feature_names()))

輸出：

vect.get_feature_names(): [u'and', u'document', u'first', u'is', u'one', u'right', u'second', u'the', u'third', u'this'] 
Traceback (most recent call last): 
    File "C:\Users\Francky\Documents\GitHub\adobe\dstc4\test\CountVectorizerSaveDic.py", line 22, in <module> 
    print('loaded_vectorizer.get_feature_names(): {0}'.format(loaded_vectorizer.get_feature_names())) 
    File "C:\Anaconda\lib\site-packages\sklearn\feature_extraction\text.py", line 890, in get_feature_names 
    self._check_vocabulary() 
    File "C:\Anaconda\lib\site-packages\sklearn\feature_extraction\text.py", line 271, in _check_vocabulary 
    check_is_fitted(self, 'vocabulary_', msg=msg), 
    File "C:\Anaconda\lib\site-packages\sklearn\utils\validation.py", line 627, in check_is_fitted 
    raise NotFittedError(msg % {'name': type(estimator).__name__}) 
sklearn.utils.validation.NotFittedError: CountVectorizer - Vocabulary wasn't fitted.

來源

2015-09-19 Franck Dernoncourt

出於某種原因，即使你通過vocabulary=vocabulary_to_load作爲參數爲sklearn.feature_extraction.text.CountVectorizer()，你仍然需要能夠調用loaded_vectorizer.get_feature_names()之前調用loaded_vectorizer._validate_vocabulary()。

在你的榜樣，因此，你應該做的與你的詞彙創建CountVectorizer對象時，以下幾點：

vocabulary_to_load = pickle.load(open(dictionary_filepath, 'r')) 
loaded_vectorizer = sklearn.feature_extraction.text.CountVectorizer(ngram_range=(ngram_size, 
             ngram_size), min_df=1, vocabulary=vocabulary_to_load) 
loaded_vectorizer._validate_vocabulary() 
print('loaded_vectorizer.get_feature_names(): {0}'. 
    format(loaded_vectorizer.get_feature_names()))

來源

2015-09-20 00:06:23

CountVectorizer：沒有安裝詞彙表

回答

相關問題