如何將SitemapSpider收集的CSV文件鏈接導入第二個蜘蛛，即CSVFeedSpider

Question

我有一個可以收集csv文件鏈接的站點地圖蜘蛛。我想用一個csv蜘蛛抓取這些鏈接。我將如何去將一隻蜘蛛的輸出提供給另一隻蜘蛛？

Answer 1

請參閱從official documentation一個例子：

from scrapy.spiders import CSVFeedSpider 
from myproject.items import TestItem 

class MySpider(CSVFeedSpider): 
    name = 'example.com' 
    allowed_domains = ['example.com'] 
    start_urls = ['http://www.example.com/feed.csv'] 
    delimiter = ';' 
    quotechar = "'" 
    headers = ['id', 'name', 'description'] 

    def parse_row(self, response, row): 
     self.logger.info('Hi, this is a row!: %r', row) 

     item = TestItem() 
     item['id'] = row['id'] 
     item['name'] = row['name'] 
     item['description'] = row['description'] 
     return item 

要使用本地文件用它來代替，只是用文件網址：file:///home/user/some.csv