13
我正在使用字袋來分類文本。它運行良好,但我想知道如何添加一個不是單詞的功能。如何添加另一個功能(文本的長度)到當前包的單詞分類? Scikit-learn
這是我的示例代碼。
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.svm import LinearSVC
from sklearn.feature_extraction.text import TfidfTransformer
from sklearn.multiclass import OneVsRestClassifier
X_train = np.array(["new york is a hell of a town",
"new york was originally dutch",
"new york is also called the big apple",
"nyc is nice",
"the capital of great britain is london. london is a huge metropolis which has a great many number of people living in it. london is also a very old town with a rich and vibrant cultural history.",
"london is in the uk. they speak english there. london is a sprawling big city where it's super easy to get lost and i've got lost many times.",
"london is in england, which is a part of great britain. some cool things to check out in london are the museum and buckingham palace.",
"london is in great britain. it rains a lot in britain and london's fogs are a constant theme in books based in london, such as sherlock holmes. the weather is really bad there.",])
y_train = [[0],[0],[0],[0],[1],[1],[1],[1]]
X_test = np.array(["it's a nice day in nyc",
'i loved the time i spent in london, the weather was great, though there was a nip in the air and i had to wear a jacket.'
])
target_names = ['Class 1', 'Class 2']
classifier = Pipeline([
('vectorizer', CountVectorizer(min_df=1,max_df=2)),
('tfidf', TfidfTransformer()),
('clf', OneVsRestClassifier(LinearSVC()))])
classifier.fit(X_train, y_train)
predicted = classifier.predict(X_test)
for item, labels in zip(X_test, predicted):
print '%s => %s' % (item, ', '.join(target_names[x] for x in labels))
現在很清楚,關於倫敦的文字往往比關於紐約的文字長得多。我將如何添加文本的長度作爲一個功能? 我是否必須使用另一種分類方式,然後結合這兩種預測?有沒有什麼辦法可以把這些文字一起做呢?一些示例代碼會很好 - 我對機器學習和scikit學習非常陌生。
您的代碼無法運行,這是因爲您只有一個目標時使用OneVsRestClassifier。 – joc
下面的鏈接幾乎完全是你使用sklearn的FeatureUnion:http://zacstewart.com/2014/08/05/pipelines-of-featureunions-of-pipelines.html – joc
看看這個答案問題http://stackoverflow.com/questions/39001956/sklearn-pipeline-transformation-on-only-certain-features/39009125#39009125 – maxymoo