2016-07-31 75 views
0

我正在嘗試使用python scrapy工具從bitcointalk.org網站提取有關用戶和公共密鑰的信息,以便他們在論壇中發佈捐贈信息。屬性錯誤響應對象沒有屬性「文本」

我在網上發現了這段代碼,對它進行了修改,以便它運行在我想要的網站上,但是我遇到了錯誤AttributeError響應對象沒有屬性文本。

下面是引用

class BitcointalkSpider(CrawlSpider): 
name = "bitcointalk" 
allowed_domains = ["bitcointalk.org"] 

start_urls = ["https://bitcointalk.org/index.php"] 

rules = (
    Rule(SgmlLinkExtractor(deny=[ 
     'https://bitcointalk\.org/index\.php\?action=ignore', 
     'https://bitcointalk\.org/index\.php\?action=profile', 
     ], 
     allow_domains='bitcointalk.org'), callback='parse_item', follow=True), 
) 

def parse_item(self, response): 
    sel = Selector(response) 
    sites = sel.xpath('//tr[contains(@class, "td_headerandpost")]') 
    items = [] 
    for site in sites: 
     item = BitcoinItem() 
     item["membername"] = site.xpath('.//td[@class="poster_info"]/b/a/text()').extract() 
     addresses = site.xpath('.//div[contains(@class, "signature")]/text()').re(r'(1[1-9A-HJ-NP-Za-km-z]{26,33})') 
     if item["membername"] and addresses: 
      addr_list = set() 
      for addr in addresses: 
       if (bcv.check_bc(addr)): 
        addr_list.add(addr) 
      item["address"] = addr_list 
      if len(addr_list) > 0: 
       items.append(item) 
    return items 

的代碼和我收到的錯誤是:

Traceback (most recent call last): 
File "/usr/local/lib/python2.7/dist-packages/scrapy/utils/defer.py", line  102, in iter_errback 
yield next(it) 
File "/usr/local/lib/python2.7/dist-packages/scrapy/spidermiddlewares/offsite.py", line 29, in process_spider_output 
for x in result: 
File "/usr/local/lib/python2.7/dist-packages/scrapy/spidermiddlewares/referer.py", line 22, in <genexpr> 
return (_set_referer(r) for r in result or()) 
File "/usr/local/lib/python2.7/dist-packages/scrapy/spidermiddlewares/urllength.py", line 37, in <genexpr> 
return (r for r in result or() if _filter(r)) 
File "/usr/local/lib/python2.7/dist-packages/scrapy/spidermiddlewares/depth.py", line 58, in <genexpr> 
return (r for r in result or() if _filter(r)) 
File "/usr/local/lib/python2.7/dist-packages/scrapy/spiders/crawl.py", line 72, in _parse_response 
cb_res = callback(response, **cb_kwargs) or() 
File "/home/sunil/Desktop/Nikhil/Thesis/mit_bitcoin/bitcoin/spiders/bitcointalk_spider.py", line 24, in parse_item 
sel = Selector(response) 
File "/usr/local/lib/python2.7/dist-packages/scrapy/selector/unified.py", line 63, in __init__ 
text = response.text 
AttributeError: 'Response' object has no attribute 'text' 

回答

0

的東西可能是錯誤的您的要求之一,因爲它似乎是從響應您的抓取至少有一個網址格式不正確。請求本身失敗,或者您沒有適當地提出請求。

See here爲您的錯誤的來源。

see here有關爲什麼您的請求格式不正確的線索。它看起來像Selector需要一個HtmlResponse對象或類似的類型。

相關問題