Scrapy中間件設置

Question

我想訪問使用scrapy的公共代理來獲取一些數據。我收到以下錯誤，當我嘗試運行代碼：

ImportError: Error loading object 'craiglist.middlewares.ProxyMiddleware': No module named middlewares

我創建middlewares.py文件與下面的代碼：

import base64 
# Start your middleware class 
class ProxyMiddleware(object): 
    # overwrite process request 
    def process_request(self, request, spider): 
     # Set the location of the proxy 
     request.meta['proxy'] = "http://124.200.36.150:8118" 

settings.py：

DOWNLOADER_MIDDLEWARES = { 
    'scrapy.contrib.downloadermiddleware.httpproxy.HttpProxyMiddleware': 110, 
    'craiglist.middlewares.ProxyMiddleware': 100, 
} 

任何幫助受到歡迎。

Answer 1

通過此設置，您需要將middlewares.py升級到craiglist包。