numpy的矩陣尺寸-TFIDF矢量

Question

林試圖解決一個聚類problem..I具有由CountVectorizer產生（）功能。本TFIDF加權向量的列表的數據類型：

<1000x5369 sparse matrix of type '<type 'numpy.float64'>' 
with 42110 stored elements in Compressed Sparse Row format> 

我具有下列尺寸的「質心」載體：

<1x5369 sparse matrix of type '<type 'numpy.float64'>' 
with 57 stored elements in Compressed Sparse Row format> 

當嘗試測量所述質心的餘弦相似性，並通過下面的行的代碼在我tfidf_vec_list其他載體：

for centroid in centroids: 
sim_scores=[cosine_similarity(vector,centroid) for vector in tfidf_vec_list] 

其中所述相似性函數是：

def cosine_similarity(vector1,vector2): 
    score=1-scipy.spatial.distance.cosine(vector1,vector2) 
    return score 

我得到錯誤：

Traceback (most recent call last): 
    File "<pyshell#25>", line 1, in <module> 
    sim_scores=[cosine_similarity(vector,centroid) for vector in tfidf_vec_list] 
    File "/home/ashwin/Desktop/Python-2.7.9/programs/test_2.py", line 28, in    cosine_similarity 
    score=1-scipy.spatial.distance.cosine(vector1,vector2) 
    File "/usr/lib/python2.7/dist-packages/scipy/spatial/distance.py", line 287, in cosine 
    dist = 1.0 - np.dot(u, v)/(norm(u) * norm(v)) 
    File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 302, in __mul__ 
    raise ValueError(**'dimension mismatch'**) 

我已經試過各種包括矩陣中的每個向量轉換成一個陣列和一個list.But我得到同樣的錯誤！

Answer 1

scipy.spatial.distance.cosine似乎不支持稀疏矩陣輸入。具體而言，np.linalg.norm（sparse_vector）失敗（請參閱Get norm of numpy sparse matrix rows）。

如果你路過它們之前都轉換輸入向量（實際上他們在這裏以矩陣形式行向量），以密集的版本，它工作得很好：

>>> xs 
<1x4 sparse matrix of type '<class 'numpy.int64'>' 
     with 3 stored elements in Compressed Sparse Row format> 
>>> ys 
<1x4 sparse matrix of type '<class 'numpy.int64'>' 
     with 3 stored elements in Compressed Sparse Row format> 
>>> cosine(xs, ys) 
Traceback (most recent call last): 
    File "<stdin>", line 1, in <module> 
    File "/usr/lib/python3.4/site-packages/scipy/spatial/distance.py", line 296, in cosine 
    dist = 1.0 - np.dot(u, v)/(norm(u) * norm(v)) 
    File "/usr/lib/python3.4/site-packages/scipy/sparse/base.py", line 308, in __mul__ 
    raise ValueError('dimension mismatch') 
ValueError: dimension mismatch 
>>> cosine(xs.todense(), ys.todense()) 
-2.2204460492503131e-16 

這應該是罰款，只有個別5369元矢量（而不是整個矩陣）。