2017-08-01 35 views
2

爲什麼scrapy.Request類調用默認的parse()方法,而我不太瞭解過程scrapy,爲什麼scrapy.Request類默認調用parse()方法?

的scrapy.Request源代碼的一部分

class Request(object_ref): 

def __init__(self, url, callback=None, method='GET', headers=None, body=None, 
      cookies=None, meta=None, encoding='utf-8', priority=0, 
      dont_filter=False, errback=None, flags=None): 

    self._encoding = encoding # this one has to be set first 
    self.method = str(method).upper() 
    self._set_url(url) 
    self._set_body(body) 
    assert isinstance(priority, int), "Request priority not an integer: %r" % priority 
    self.priority = priority 

    assert callback or not errback, "Cannot use errback without a callback" 
    self.callback = callback 
    self.errback = errback 

....

但這種默認的回調是沒有讓我這個

if "msg" in text_json and text_json["msg"] == "login": 
     for url in self.start_urls: 
      yield scrapy.Request(url, dont_filter=True, headers=self.headers) 

回答

0

很疑惑這是這是decided inside the Scrapy core,看到這個request.callback or spider.parse部分:

def call_spider(self, result, request, spider): 
    result.request = request 
    dfd = defer_result(result) 
    dfd.addCallbacks(request.callback or spider.parse, request.errback) 
    return dfd.addCallback(iterate_spider_output)