2017-01-19 37 views
0

我在sqlite3數據庫中有數百萬行。單行的列「點」包含了下面的例子:在單列中搜索950個數字的最佳方法是什麼?

{1399808086,1366221142,1374614902,1374608759,1375598069,1375270116,1935207612,1914502332,1913478333,1930188205,1934563311,1942881023,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988 ... up to nearly 1'000 numbers, ending with } 

什麼是搜索行,其中列「點」包括最好的辦法
一)所有三個示例搜索值 1366221142,1374614902 ,1374608759
(3位上有兩個,並且在上面的內容爲四個)

b)中儘可能多地(3或2或1)的上述3示例性搜索值

我與索引試過,但一用LIKE搜索和'%1366221142%'需要「永遠」。

其實我嘗試它FTS5,但導入到新創建的虛擬表似乎需要幾天的時間。

你知道任何其他可能性?

+0

最好的辦法是將數據標準化爲每行一個數字的第二個表。 –

+0

每行一個號碼?幾百萬行×950的數字會變得很大,而不是? – PeterCo

+0

它將包含與當前數據庫完全一樣多的數字。 –

回答

1

如果你可以使用Python那麼這種做法是「非常快」。

import sqlite3 
conn = sqlite3.connect(':memory:') 

points = '{1399808086,1366221142,1374614902,1374608759,1375598069,1375270116,1935207612,1914502332,1913478333,1930188205,1934563311,1942881023,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860,1929778348,1900414380,1883651988,1373508175,-778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,-778051210,-740437658,-749943514,-754136794,-770946570,1376606678,1380850053,1380854148,1381902468,1381971092,308228244,324940180,324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343,1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860}' 

conn.execute('CREATE TABLE something (recno, points)') 

for r in range(10000): 
    conn.execute('INSERT INTO something (recno, points) values (?,?)', (r, points)) 

seeking = ['1366221142', '1374614902', '1374608759'] 
first = True 
for row in conn.execute('SELECT recno, points FROM something'): 
    pointsList = points[1:-1].split(',') 
    counts = { _:pointsList.count(_) for _ in seeking } 
    if first: 
     print (row) 
     print (counts) 
     first = False 

的(刪節)輸出是:

(0,「{1399808086,1366221142,1374614902,1374608759,1375598069,1375270116,1935207612,1914502332,1913478333,1930188205,1934563311,1942881023,1373508175, - 778100129,-788765075,-788763091,-790856156,-790835404,-791756027,-795938489,-779165370,... 324948372,327078869,292409717,275550503,275554606,275547438,812554558,812489022,1894554398,1895733774,1895741966,1912515343, 1943993629,1935471709,1918694493,1914490972,1913409788,1913475260,1913458860} ') {' 1374614902' :1, '1374608759':1, '1366221142':1}

請注意,代碼會將您的字符串的10,000個副本放到數據庫中,並將其擴展到1,000個數字,然後對其進行處理。當然,數據庫在內存中,這是一個需要考慮的因素。

你可以只嘗試。

相關問題