當我運行Nutch並且一個鏈接不再存在時,我可以運行readdb
命令,它顯示有一些URL標記爲db_gone
。SolrClean不是從Solr刪除
於是我運行SolrClean命令和它說:
SolrClean deleting a total of 1 documents
哪個是正確的,但沒有從Solr的
幫助去除?
如果你想看看我的配置,然後我有我自己的Solr/Nutch的設置是如何配置的Here
編輯一個博客:
這是一個很好的機會,它是不只是沒有工作的SolrClean命令,我有一種感覺,這與我的設置有關,刪除沒有被提交。
這是文件發出的刪除請求 - 但該文件存在:
INFO - 2013-08-09 15:54:52.729;
org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/solr path=/update params={wt=javabin&version=2}
{delete=file:/C:/Users/alamil/Documents/TextFiles/Y2012.doc
(-1442903587791306752)]} 0 2
這是整個日誌:crawldb的
INFO - 2013-08-09 15:54:51.785; org.apache.solr.search.SolrIndexSearcher; Opening [email protected] main
INFO - 2013-08-09 15:54:51.786; org.apache.solr.core.QuerySenderListener; QuerySenderListener sending requests to [email protected] main{StandardDirectoryReader(segments_7w:549:nrt _6j(4.3.1):C12/11 _6k(4.3.1):C12)}
INFO - 2013-08-09 15:54:51.787; org.apache.solr.core.QuerySenderListener; QuerySenderListener done.
INFO - 2013-08-09 15:54:51.787; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
INFO - 2013-08-09 15:54:51.788; org.apache.solr.core.SolrCore; [collection1] Registered new searcher [email protected] main{StandardDirectoryReader(segments_7w:549:nrt _6j(4.3.1):C12/11 _6k(4.3.1):C12)}
INFO - 2013-08-09 15:54:51.789; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&waitFlush=true&version=2} {commit=} 0 903
INFO - 2013-08-09 15:54:52.053; org.apache.solr.core.SolrCore; [collection1] webapp=/solr path=/select params={fl=id&q=id:[*+TO+*]&wt=javabin&version=2&rows=1} hits=13 status=0 QTime=1
INFO - 2013-08-09 15:54:52.355; org.apache.solr.core.SolrCore; [collection1] webapp=/solr path=/select params={fl=id&q=id:[*+TO+*]&wt=javabin&version=2&rows=1} hits=13 status=0 QTime=0
INFO - 2013-08-09 15:54:52.413; org.apache.solr.core.SolrCore; [collection1] webapp=/solr path=/select params={fl=id,boost,tstamp,digest&start=0&q=id:[*+TO+*]&wt=javabin&version=2&rows=13} hits=13 status=0 QTime=1
INFO - 2013-08-09 15:54:52.729; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {delete=[file:/C:/Users/alamil/Documents/TextFiles/Y2012.doc (-1442903587791306752)]} 0 2
INFO - 2013-08-09 15:54:52.733; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
INFO - 2013-08-09 15:54:52.835; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
commit{dir=NRTCachingDirectory([email protected]:\Users\alamil\Documents\Test\solr_home\data\index [email protected]; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_7w,generation=284,filenames=[_6j_1.del, _6k_Lucene41_0.pos, segments_7w, _6j.nvd, _6j_Lucene41_0.tim, _6j_Lucene41_0.tip, _6k.fdt, _6k.fnm, _6j_Lucene41_0.pos, _6j.nvm, _6k_Lucene41_0.doc, _6k_Lucene41_0.tim, _6k.si, _6j.si, _6k.nvd, _6k.fdx, _6j_Lucene41_0.doc, _6j.fdt, _6k.nvm, _6j.fdx, _6k_Lucene41_0.tip, _6j.fnm]
commit{dir=NRTCachingDirectory([email protected]:\Users\alamil\Documents\Test\solr_home\data\index [email protected]; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_7x,generation=285,filenames=[_6j_1.del, _6k_Lucene41_0.pos, _6j.nvd, _6j_Lucene41_0.tim, _6j_Lucene41_0.tip, _6k.fdt, _6k.fnm, _6j_Lucene41_0.pos, _6j.nvm, _6k_Lucene41_0.doc, _6k_Lucene41_0.tim, _6k.si, _6j.si, _6k.nvd, _6k.fdx, segments_7x, _6j_Lucene41_0.doc, _6j.fdt, _6k.nvm, _6j.fdx, _6k_Lucene41_0.tip, _6j.fnm]
INFO - 2013-08-09 15:54:52.835; org.apache.solr.core.SolrDeletionPolicy; newest commit = 285[_6j_1.del, _6k_Lucene41_0.pos, _6j.nvd, _6j_Lucene41_0.tim, _6j_Lucene41_0.tip, _6k.fdt, _6k.fnm, _6j_Lucene41_0.pos, _6j.nvm, _6k_Lucene41_0.doc, _6k_Lucene41_0.tim, _6k.si, _6j.si, _6k.nvd, _6k.fdx, segments_7x, _6j_Lucene41_0.doc, _6j.fdt, _6k.nvm, _6j.fdx, _6k_Lucene41_0.tip, _6j.fnm]
我的猜測是,變化不是'commited',我知道[有](https://issues.apache.org/jira/browse/NUTCH-1000)作爲選項進行提交的票據 –
沒有標誌提交該操作,因此如果它沒有提交,我不知道該怎麼做 –