的urlopen HTTP錯誤

我嘗試打開由古騰堡項目頁面編輯與BeautifulSoup的urlopen HTTP錯誤

import urllib2 
from bs4 import BeautifulSoup 

url = "http://www.gutenberg.org/files/54801/54801-h/54801-h.htm" 
page = urllib2.urlopen(url) 
soup_packtpage=BeautifulSoup(page) 

print(soup_packtpage)

我CLOUD9工作。我有以下錯誤：

Traceback (most recent call last): 
File "soup.py", line 5, in <module> 
page = urllib2.urlopen(url) 
File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen 
return _opener.open(url, data, timeout) 
File "/usr/lib/python2.7/urllib2.py", line 410, in open 
response = meth(req, response) 
File "/usr/lib/python2.7/urllib2.py", line 523, in http_response 
'http', request, response, code, msg, hdrs) 
File "/usr/lib/python2.7/urllib2.py", line 448, in error 
return self._call_chain(*args) 
File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain 
result = func(*args) 
File "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default 
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) 
urllib2.HTTPError: HTTP Error 403: Forbidden

出了什麼問題？

來源

2017-05-30 Lucas

這是一個HTTP錯誤，而不是python錯誤。這是說你不能提出這樣的要求。可能缺少標頭，如Cookie或API憑證 – adelineu

我無法重現此問題。我得到一個巨大的打印輸出。 – roganjosh

你得到403，因爲它需要設置cookie。我第一次嘗試就把我帶到〜welcome_stranger頁面，然後再次嘗試請求成功。這是你看到的東西嗎？ – oshaiken

您應該嘗試使用請求包。

這工作正常，我在Python 3.6

import requests 
from bs4 import BeautifulSoup as bs4 

url = "http://www.gutenberg.org/files/54801/54801-h/54801-h.htm" 
r = requests.get(url) 
#page = urllib3.urlopen(url) 
soup_packtpage = bs4(r.text, 'html.parser') 

print(soup_packtpage) 

paragrapghs = soup_packtpage.findAll("p") 
print(paragrapghs) 

f = open("guttenberg_book.html", 'a', encoding="utf-8") 
f.write(str(paragrapghs)) 
f.close()

我使用BS4，讓你開始..這只是輸出書文本增加了打印款.. :)

來源

2017-05-30 15:27:28 johnashu

import cookielib 
import urllib2 

headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12;rv:50.0) Gecko/20100101 Firefox/50.0'} 
cookie = cookielib.CookieJar() 
handler = urllib2.HTTPCookieProcessor(cookie) 
opener = urllib2.build_opener(handler) 

request = urllib2.Request(url = "http://www.gutenberg.org/files/54801/54801-h/54801-h.htm", headers=headers) 
page = opener.open(request).read()

嘗試請求並添加標題。它適用於Python 2.7.13

來源

2017-05-30 16:08:08

的urlopen HTTP錯誤

回答

相關問題