類型錯誤：to_bytes必須接受一個Unicode，STR或字節的對象，得到了設置

Question

我使用Scrapy寫的蜘蛛，但我遇到這個錯誤。這裏是我的代碼：

# -*- coding: utf-8 -*- 
import scrapy 

class ZhihuSpider(scrapy.Spider): 
    name = "zhihu" 
    allowed_domains = ["www.zhihu.com"] 

    def start_requests(self): 
     return [scrapy.Request('http://www.zhihu.com/#signin')] 

    def parse(self, response): 
     print response 

錯誤的信息：

Traceback (most recent call last): 
    File "C:\Python27\lib\site-packages\twisted\internet\defer.py", line 1183, in 
_inlineCallbacks 
    result = result.throwExceptionIntoGenerator(g) 
    File "C:\Python27\lib\site-packages\twisted\python\failure.py", line 389, in t 
hrowExceptionIntoGenerator 
    return g.throw(self.type, self.value, self.tb) 
    File "C:\Python27\lib\site-packages\scrapy\core\downloader\middleware.py", lin 
e 37, in process_request 
    response = yield method(request=request, spider=spider) 
    File "C:\Python27\lib\site-packages\twisted\internet\defer.py", line 587, in _ 
runCallbacks 
    current.result = callback(current.result, *args, **kw) 
    File "C:\Python27\lib\site-packages\scrapy\downloadermiddlewares\robotstxt.py" 
, line 45, in process_request_2 
    to_native_str(self._useragent), request.url): 
    File "C:\Python27\lib\site-packages\scrapy\utils\python.py", line 127, in to_n 
ative_str 
    return to_bytes(text, encoding, errors) 
    File "C:\Python27\lib\site-packages\scrapy\utils\python.py", line 117, in to_b 
ytes 
    'object, got %s' % type(text).__name__) 
TypeError: to_bytes must receive a unicode, str or bytes object, got set 

Answer 1

當您嘗試使用錯誤的數據類型時，可能會遇到錯誤。 例如：

str = 15 
print str.encode("ascii") # Error occurs 

str = "15" 
print str.encode("ascii") # Right code because encode function belongs to unicode string data type but not integer. 

Answer 2

你allowed_domains是無效的，這樣

allowed_domains = ["zhihu.com"] 

在scrapy.core.downloader.webclient.py，每個URL將被解析。函數to_bytes將檢查它是否爲None，否則將引發TypeError。