使用Python實現主題模型（numpy）

Question

最近，我在Python上使用numpy實現了針對LDA主題模型的Gibbs採樣，並將其作爲參考來源於站點的一些代碼。在吉布斯採樣的每次迭代中，我們刪除一個（當前）字，根據LDA模型推斷的後驗條件概率分佈爲該字採樣一個新話題，並更新字 - 話題計數，如下所示：

for m, doc in enumerate(docs): #m: doc id 
    for n, t in enumerate(doc): #n: id of word inside document, t: id of the word globally 
    # discount counts for word t with associated topic z 
    z = z_m_n[m][n] 
    n_m_z[m][z] -= 1 
    n_z_t[z, t] -= 1 
    n_z[z] -= 1 
    n_m[m] -= 1 

    # sample new topic for multinomial     
    p_z_left = (n_z_t[:, t] + beta)/(n_z + V * beta) 
    p_z_right = (n_m_z[m] + alpha)/(n_m[m] + alpha * K) 
    p_z = p_z_left * p_z_right 
    p_z /= numpy.sum(p_z) 
    new_z = numpy.random.multinomial(1, p_z).argmax() 

    # set z as the new topic and increment counts 
    z_m_n[m][n] = new_z 
    n_m_z[m][new_z] += 1 
    n_z_t[new_z, t] += 1 
    n_z[new_z] += 1 
    n_m[m] += 1 

在上面的代碼中，我們用多項式scipy函數對新的（單個）z進行了採樣。

現在，我想實現一個聯合情緒主題模型this paper。現在，我需要保持跟蹤所需要的計數以下結構：

3D matrix containing # occurrences for a word for each topic, for each sentiment 
3D matrix containing # occurrences for a topic, for each sentiment, for each document 
2D matrix containing # occurrences for a topic, for each sentiment 
2D matrix containing # occurrences for a sentiment for each document 

現在問題來了：在這個吉布斯採樣器，爲一個文件既是一個新的話題和情緒標籤是在看到每個字現在從條件後驗樣本中抽取（本文的第4頁公式5）。 現在我怎樣才能在Python中「採樣這兩個值」？

在此先感謝...

Answer 1

試試這個。從關於主題和情感標籤的聯合分佈抽樣意味着整個T×S矩陣應該總計爲1.

docs=[[0,1],[0,0],[1,0,1]] 
D=len(docs) 
z_d_n=[[0 for _ in xrange(len(d))] for d in docs] 
l_d_n=[[0 for _ in xrange(len(d))] for d in docs] 

V=2 
T=2 
S=2 
n_m_j_k=numpy.zeros((V,T,S)) 
n_j_k_d=numpy.zeros((T,S,D)) 
n_j_k=numpy.zeros((T,S)) 
n_k_d=numpy.zeros((S,D)) 
n_d=numpy.zeros((D)) 

beta=.1 
alpha=.1 
gamma=.1 

for d, doc in enumerate(docs): #d: doc id 
    for n, m in enumerate(doc): #i: index of the word inside document, m: id of the word in the vocabulary 
     # j is the topic 
     j = z_d_n[d][n] 
     # k is the sentiment 
     k = l_d_n[d][n] 
     n_m_j_k[m][j][k] += 1 
     n_j_k_d[j][k][d] += 1 
     n_j_k[j][k] += 1 
     n_k_d[k][d] += 1 
     n_d[d] += 1 

for d, doc in enumerate(docs): #d: doc id 
    for n, m in enumerate(doc): #i: index of the word inside document, m: id of the word in the vocabulary 
     # j is the topic 
     j = z_d_n[d][n] 
     # k is the sentiment 
     k = l_d_n[d][n] 
     n_m_j_k[m][j][k] -= 1 
     n_j_k_d[j][k][d] -= 1 
     n_j_k[j][k] -= 1 
     n_k_d[k][d] -= 1 
     n_d[d] -= 1 

     # sample a new topic and sentiment label jointly 
     # T is the number of topics 
     # S is the number of sentiments 
     p_left = (n_m_j_k[m] + beta)/(n_j_k + V * beta) # T x S array 
     p_mid = (n_j_k_d[:,:,d] + alpha)/numpy.tile(n_k_d[:,d] + T * alpha, (T,1)) 
     p_right = numpy.tile(n_k_d[:,d] + gamma,(T,1))/numpy.tile(n_d[d] + S * gamma,(T,S)) 
     p = p_left * p_mid * p_right 
     p /= numpy.sum(p) 
     new_jk = numpy.random.multinomial(1, numpy.reshape(p, (T*S))).argmax() 
     j=new_jk/T 
     k=new_jk%T 

     z_d_n[d][n]=j 
     l_d_n[d][n]=k 
     n_m_j_k[m][j][k] += 1 
     n_j_k[j][k] += 1 
     n_k_d[k][d] += 1 
     n_d[d] += 1