檢查輸入文本中來自單詞集合的單詞

Question

我收集了名詞短語約10,000個單詞。我想檢查這些NP收集的每個新的輸入文本數據，並提取那些包含這些NP的任何句子。我不想爲每個單詞運行循環，因爲這會使我的代碼變得很慢。我正在使用Java和Stanford CoreNLP。

Answer 1

一個快速簡便的方法是使用Regexner來識別字典中任何東西的所有示例，然後檢查句子中的非「O」NER標籤。

package edu.stanford.nlp.examples; 

import edu.stanford.nlp.ling.*; 
import edu.stanford.nlp.pipeline.*; 
import edu.stanford.nlp.util.*; 
import java.util.*; 
import java.util.stream.Collectors; 

public class FindSentencesWithPhrase { 

    public static boolean checkForNamedEntity(CoreMap sentence) { 
    for (CoreLabel token : sentence.get(CoreAnnotations.TokensAnnotation.class)) { 
     if (token.ner() != null && !token.ner().equals("O")) { 
     return true; 
     } 
    } 
    return false; 
    } 

    public static void main(String[] args) { 
    Properties props = new Properties(); 
    props.setProperty("annotators", "tokenize,ssplit,pos,lemma,regexner"); 
    props.setProperty("regexner.mapping", "phrases.rules"); 
    StanfordCoreNLP pipeline = new StanfordCoreNLP(props); 
    String exampleText = "This sentence contains the phrase \"ice cream\"." + 
     "This sentence is not of interest. This sentences contains pizza."; 
    Annotation ann = new Annotation(exampleText); 
    pipeline.annotate(ann); 
    for (CoreMap sentence : ann.get(CoreAnnotations.SentencesAnnotation.class)) { 
     if (checkForNamedEntity(sentence)) { 
     System.out.println("---"); 
     System.out.println(sentence.get(CoreAnnotations.TokensAnnotation.class). 
      stream().map(token -> token.word()).collect(Collectors.joining(" "))); 
     } 
    } 
    } 
} 

文件「phrases.rules」應該是這樣的：

ice cream  PHRASE_OF_INTEREST  MISC 1 
pizza PHRASE_OF_INTEREST  MISC 1