NLTK FreqDist，繪製標準化計數？

Question

在NLTK，你可以很容易地計算的話計數文本，比方說，通過做

from nltk.probability import FreqDist 
fd = FreqDist([word for word in text.split()]) 

，其中文本是一個字符串。 現在，你可以繪製分佈

fd.plot() 

，這將讓你與計數每個字一個很好的線圖。在docs中沒有提到繪製實際頻率的方法，您可以在fd.freq(x)中看到。

繪製標準化計數的任何直接方法，不需要將數據轉化爲其他數據結構，分別標準化和繪圖？

Answer 1

您可以更新FD [文字]與FD [文字] /總

from nltk.probability import FreqDist 

text = "This is an example . This is test . example is for freq dist ." 
fd = FreqDist([word for word in text.split()]) 

total = fd.N() 
for word in fd: 
    fd[word] /= float(total) 

fd.plot() 

注意：您將失去原始FreqDist值。

Answer 2

請原諒缺少文件。在nltk,FreqDist爲您提供文本中的原始計數（即單詞的頻率），但ProbDist爲您提供給定文本的單詞的概率。

欲瞭解更多信息，你必須做一些讀碼：https://github.com/nltk/nltk/blob/develop/nltk/probability.py

的具體線路是做正常化自帶形式https://github.com/nltk/nltk/blob/develop/nltk/probability.py#L598

因此，要獲得一個標準化ProbDist，你可以做到以下幾點：

>>> from nltk.corpus import brown 
>>> from nltk.probability import FreqDist 
>>> from nltk.probability import DictionaryProbDist 
>>> brown_freqdist = FreqDist(brown.words()) 
# Cast the frequency distribution into probabilities 
>>> brown_probdist = DictionaryProbDist(brown_freqdist) 
# Something strange in NLTK to note though 
# When asking for probabilities in a ProbDist without 
# normalization, it looks it returns the count instead... 
>>> brown_freqdist['said'] 
1943 
>>> brown_probdist.prob('said') 
1943 
>>> brown_probdist.logprob('said') 
10.924070185585345 
>>> brown_probdist = DictionaryProbDist(brown_freqdist, normalize=True) 
>>> brown_probdist.logprob('said') 
-9.223104921442907 
>>> brown_probdist.prob('said') 
0.0016732805599763002