FORMSOF的SQL Server全文搜索條件排除停用詞

Question

我想搜索一些帶停用詞的短語，例如「Line Through Crack」。 「通過」是停用詞。我希望得到同樣的結果類似查詢

CONTAINS(*, 'FORMSOF(INFLECTIONAL, "Line") AND FORMSOF(INFLECTIONAL, "Crack")') 

所以，它包含了所有形式的所有詞，除了停用詞的所有行。 如果客戶不知道停用詞列表，我可以這樣做嗎？

Answer 1

您正在使用什麼版本的SQL Server？如果是2008或更高版本，則可以在查詢運行時以編程方式檢索停用詞列表。然後，您可以檢查是否有任何搜索項位於停用詞列表中，並將它們從「CONTAINS」查詢字符串中排除。

下面的查詢將返回的停止詞列表（美國英語，這是語言ID 1033）：

-- Run the following to get a list of languages and their IDs 
select lcid, name from sys.syslanguages order by 1 

-- Then use that ID to get a list of stop words 
select * from sys.fulltext_stopwords where language_id = 1033 

有了這些信息，你可以寫一個搜索PROC做這樣的事情（這是一個非常簡單的例子，但你應該明白我的意思）：

USE [AdventureWorks] 
GO 
-- Make sure you have a full-text catalogue to test against 
/* 
IF EXISTS(SELECT * FROM sys.fulltext_indexes WHERE [object_id] = OBJECT_ID('Production.ProductDescription')) 
    DROP FULLTEXT INDEX ON Production.ProductDescription; 
IF EXISTS(SELECT * FROM sys.fulltext_catalogs WHERE name = 'FTC_product_description') 
    DROP FULLTEXT CATALOG FTC_product_description; 
CREATE FULLTEXT CATALOG [FTC_product_description] 
    WITH ACCENT_SENSITIVITY = OFF 
    AS DEFAULT AUTHORIZATION [dbo] 
CREATE FULLTEXT INDEX ON [Production].[ProductDescription]([Description] LANGUAGE [English]) 
    KEY INDEX [PK_ProductDescription_ProductDescriptionID] ON ([FTC_product_description], FILEGROUP [PRIMARY]) 
    WITH (CHANGE_TRACKING = AUTO, STOPLIST = SYSTEM); 
*/ 
GO 
IF OBJECT_ID('dbo.my_search_proc') IS NULL EXEC ('CREATE PROC dbo.my_search_proc AS '); 
GO 
-- My Search Proc 
ALTER PROC dbo.my_search_proc (
    @query_string NVARCHAR(1000), 
    @language_id INT = 1033 -- change this to whatever your default language ID is 
) AS 
BEGIN 
    SET NOCOUNT ON; 

    ------------------------------------------------------ 
    -- Split the string into 1 row per word 
    ------------------------------------------------------ 
    -- I've done this in-line here for simplicity, but I 
    -- would recommend creating a CLR function instead 
    -- for performance reasons. 
    DECLARE @words TABLE (id INT IDENTITY(1,1), word NVARCHAR(100)); 
    DECLARE @cnt INT, @split_on CHAR(1) 
    SELECT @cnt = 1, @split_on = ' '; 
    WHILE (CHARINDEX(@split_on, @query_string) > 0) 
    BEGIN 
     INSERT INTO @words (word) 
     SELECT word = LEFT(LTRIM(RTRIM(SUBSTRING(@query_string,1,CHARINDEX(@split_on,@query_string)-1))), 100); 
     SET @query_string = SUBSTRING(@query_string,CHARINDEX(@split_on,@query_string)+1,LEN(@query_string)); 
     SET @cnt = @cnt + 1; 
    END 
    INSERT INTO @words (word) 
    SELECT word = LEFT(LTRIM(RTRIM(@query_string)), 100); 

    ------------------------------------------------------ 
    -- Now build your "FORMSOF" string, excluding stop words. 
    ------------------------------------------------------ 
    DECLARE @formsof NVARCHAR(4000); 

    SELECT @formsof = ISNULL(@formsof, '') 
      + 'FORMSOF(INFLECTIONAL, "' + w.word + '") AND ' 
    FROM @words AS w 
    LEFT JOIN sys.fulltext_system_stopwords AS sw -- use sys.fulltext_stopwords instead if you're using a user-defined stop-word list (or use both) 
      ON w.word = sw.stopword COLLATE database_default 
      AND sw.language_id = @language_id 
    WHERE sw.stopword IS NULL 
    ORDER BY w.id; -- retain original order in case you do any weighting based on position, etc. 

    -- If nothing was returned, then the whole query string was made up of stop-words, 
    -- so just return an empty result set to the application. 
    IF @@ROWCOUNT = 0 
     SELECT TOP(0) * FROM Production.ProductDescription; 

    SET @formsof = LEFT(@formsof, LEN(@formsof)-4); -- Remove the last "AND" 
    PRINT 'Query String: ' + @formsof 

    ------------------------------------------------------ 
    -- Now perform the actual Full-Text search 
    ------------------------------------------------------ 
    SELECT * 
    FROM Production.ProductDescription 
    WHERE CONTAINS(*, @formsof); 
END 
GO 

EXEC dbo.my_search_proc 'bars for downhill'; 

所以，如果你搜索「下坡棒」，那麼「爲」將被清除掉（因爲其停止字） ，你應該留下FORMSOF(INFLECTIONAL, "bars") AND FORMSOF(INFLECTIONAL, "downhill").

不幸的是，如果你使用的是SQL 2005並且不知道噪音詞文件中有什麼，那麼你可以做的事情並不多（據我所知）。

乾杯， 戴夫