VB.NET - 針對另一個列表的大型通用列表篩選

Question

我有兩個字符串類型的通用列表，第一個包含大約1,000,000個術語，第二個包含大約100,000個關鍵字。第一個列表中的術語可能包含也可能不包含第二個列表中的關鍵字。我需要在第一個列表中隔離那些不包含第二個列表中的任何關鍵字的術語。 目前我正在做這樣的（VB.NET與框架3.5）：

For Each keyword In keywordList 
    termList.RemoveAll(AddressOf ContainsKeyword) 
Next 

Private Shared Function ContainsKeyword(ByVal X As String) As Integer 
    If X.IndexOf(keyword) >= 0 Then 
     Return True 
    Else 
     Return False 
    End If 
End Function 

不用說，這需要永遠。什麼是最快的方法來完成這個？也許使用字典？任何提示將有所幫助

Answer 1

一個關鍵字的字典將不會在這裏工作，因爲你在做Contains檢查，而不僅僅是直接相等檢查。您可能採取的一種方法是將搜索條件組合到樹中。樹幫助的數量取決於搜索條件中有多少重疊。我組建了一個基本的樹的實現（沒有太多的測試）爲起點：

Public Class WordSearchTree 

    Private ReadOnly _branches As New Dictionary(Of Char, WordSearchTree) 

    Public Function WordContainsTerm(ByVal word As String) As Boolean 
     Return Not String.IsNullOrEmpty(word) AndAlso _ 
       Enumerable.Range(0, word.Length - 1) _ 
         .Any(Function(i) WordContainsInternal(word, i)) 
    End Function 

    Private Function WordContainsInternal(ByVal word As String, ByVal charIndex As Integer) As Boolean 
     Return _branches.Count = 0 OrElse _ 
       (_branches.ContainsKey(word(charIndex)) AndAlso _ 
       charIndex < word.Length - 1 AndAlso _ 
       _branches(word(charIndex)).WordContainsInternal(word, charIndex + 1)) 
    End Function 

    Public Shared Function BuildTree(ByVal words As IEnumerable(Of String)) As WordSearchTree 
     If words Is Nothing Then Throw New ArgumentNullException("words") 
     Dim ret As New WordSearchTree() 
     For Each w In words 
      Dim curTree As WordSearchTree = ret 
      For Each c In w 
       If Not curTree._branches.ContainsKey(c) Then 
        curTree._branches.Add(c, New WordSearchTree()) 
       End If 
       curTree = curTree._branches(c) 
      Next 
     Next 
     Return ret 
    End Function 

End Class 

，並與樹，你可以做這樣的事情：

Dim keys As WordSearchTree = WordSearchTree.Build(keywordList) 
termList.RemoveAll(AddressOf keys.WordContainsTerm)