Scrapy + Splash（Docker）問題

Question

我在AWS Ubuntu服務器上設置了scrapy和scrapy-splash。它在一段時間內工作正常，但幾個小時後，我會開始得到這樣的錯誤消息;

Traceback (most recent call last): 
    File "/home/ubuntu/.local/lib/python3.5/site- 
packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks 
result = result.throwExceptionIntoGenerator(g) 
    File "/home/ubuntu/.local/lib/python3.5/site- 
packages/twisted/python/failure.py", line 393, in throwExceptionIntoGe 
nerator 
    return g.throw(self.type, self.value, self.tb) 
    File "/home/ubuntu/.local/lib/python3.5/site- 
packages/scrapy/core/downloader/middleware.py", line 43, in process_re 
quest 
defer.returnValue((yield download_func(request=request,spider=spider))) 
twisted.internet.error.ConnectionRefusedError: Connection was refused by 
other side: 111: Connection refused. 

我會發現docker中的splash進程已終止或無響應。

我一直在運行着飛濺過程;

sudo docker run -p 8050:8050 scrapinghub/splash 

根據scrapy-splash說明。

我試着在tmux shell中啓動進程以確保ssh連接不會干擾splah進程，但沒有運氣。

想法？

Answer 1

您應該運行容器--restart和-d選項。請參閱documentation如何在生產中運行Splash。