如何處理餘弦相似度的負值

Question

我根據條款計算了我的文檔的tf-idf。然後，我應用LSA來降低條款的維度。 'similarity_dist'包含負值（參見下表）。我如何計算範圍0-1的餘弦距離？

tf_vectorizer = CountVectorizer(max_df=0.95, min_df=2, tokenizer=tokenize_and_stem, stop_words='english') 
%time tf = tf_vectorizer.fit_transform(descriptions) 
print(tf.shape) 
svd = TruncatedSVD(100) 
normalizer = Normalizer(copy=False) 
lsa = make_pipeline(svd, normalizer) 
tfidf_desc = lsa.fit_transform(tfidf_matrix_desc) 
explained_variance = svd.explained_variance_ratio_.sum() 
print("Explained variance of the SVD step: {}%".format(int(explained_variance * 100))) 

similarity_dist = cosine_similarity(tfidf_desc) 
pd.DataFrame(similarity_dist,index=descriptions.index, columns=descriptions.index).head(10) 

print(tfidf_matrix_desc.min(),tfidf_matrix_desc.max()) 
#0.0 0.736443429828 

print(tfidf_desc.min(),tfidf_desc.max()) 
#-0.518015429416 0.988306783341 

print(similarity_dist.max(),similarity_dist.min()) 
#1.0 -0.272010919022 

Answer 1

cosine_similarity是在-1到1

餘弦距離的範圍被定義爲：

cosine_distance = 1 - cosine_similarity 

因此cosine_distance將在的範圍：0至2

請參閱https://en.wikipedia.org/wiki/Cosine_similarity

餘弦距離是一個常用於正空間補數的術語，即：D_C（A，B）= 1 - S_C（A，B）。

注：如果你必須有它在0到1的範圍內，可以使用cosine_distance/2