爲什麼NLTK Lemmatizer不能解釋一些複數單詞？

Question

我試圖從古蘭經聖書中解讀一些詞，但有些詞不能詞法化。

這裏是我的一句話：

sentence = "Then bring ten surahs like it that have been invented and call upon for assistance whomever you can besides Allah if you should be truthful" 

那句話是我的txt數據集的一部分。你可以看到有 ，有「surah」這是複數形式的「surah」。 我已經盡我的代碼：

def lemmatize(self, ayat): 
    wordnet_lemmatizer = WordNetLemmatizer() 
    result = [] 

    for i in xrange (len(ayat)): 
     result.append(wordnet_lemmatizer.lemmatize(sentence[i],'v')) 
    return result 

，當我運行和打印，結果是這樣的：

['bring', 'ten', 'surahs', 'like', u'invent', 'call', 'upon', 'assistance', 'whomever', 'besides', 'Allah', 'truthful'] 

的「surahs」不變成「古蘭經」。

有人可以說爲什麼？謝謝。

Answer 1

見

• Stemming some plurals with wordnet lemmatizer doesn't work
• Python NLTK Lemmatization of the word 'further' with wordnet

對於大多數非標英語單詞，共發現Lemmatizer沒有什麼幫助中得到正確的引理，嘗試一個詞幹：

>>> from nltk.stem import PorterStemmer 
>>> porter = PorterStemmer() 
>>> porter.stem('surahs') 
u'surah' 

---

此外，嘗試lemmatize_sent在earthy（一nltk包裝，「無恥插頭」）：

>>> from earthy.nltk_wrappers import lemmatize_sent 
>>> sentence = "Then bring ten surahs like it that have been invented and call upon for assistance whomever you can besides Allah if you should be truthful" 
>>> lemmatize_sent(sentence) 
[('Then', 'Then', 'RB'), ('bring', 'bring', 'VBG'), ('ten', 'ten', 'RP'), ('surahs', 'surahs', 'NNS'), ('like', 'like', 'IN'), ('it', 'it', 'PRP'), ('that', 'that', 'WDT'), ('have', 'have', 'VBP'), ('been', u'be', 'VBN'), ('invented', u'invent', 'VBN'), ('and', 'and', 'CC'), ('call', 'call', 'VB'), ('upon', 'upon', 'NN'), ('for', 'for', 'IN'), ('assistance', 'assistance', 'NN'), ('whomever', 'whomever', 'NN'), ('you', 'you', 'PRP'), ('can', 'can', 'MD'), ('besides', 'besides', 'VB'), ('Allah', 'Allah', 'NNP'), ('if', 'if', 'IN'), ('you', 'you', 'PRP'), ('should', 'should', 'MD'), ('be', 'be', 'VB'), ('truthful', 'truthful', 'JJ')] 

>>> words, lemmas, tags = zip(*lemmatize_sent(sentence)) 
>>> lemmas 
('Then', 'bring', 'ten', 'surahs', 'like', 'it', 'that', 'have', u'be', u'invent', 'and', 'call', 'upon', 'for', 'assistance', 'whomever', 'you', 'can', 'besides', 'Allah', 'if', 'you', 'should', 'be', 'truthful') 

>>> from earthy.nltk_wrappers import pywsd_lemmatize 
>>> pywsd_lemmatize('surahs') 
'surahs' 

>>> from earthy.nltk_wrappers import porter_stem 
>>> porter_stem('surahs') 
u'surah'