用Python刮臉BeautifulSoup 4

Question

我對Webscraping相當陌生，在網上看了一些教程視頻之後給了我一個嘗試。我決定使用Tripadvisor.com並嘗試從顧客評論收集數據。

這是我想出了（代碼）：

from urllib.request import urlopen as uReq 

from bs4 import BeautifulSoup as soup 

my_url = 'https://www.tripadvisor.com.sg/Attraction_Review-g293916-d12033454- 
Reviews-SHOW_DC-Bangkok.html' 

uClient = uReq(my_url) 
page_html = uClient.read() 
uClient.close() 

page_soup = soup(page_html, "html.parser") 

containers = page_soup.findAll("div",{"class":"ui_column is-9"}) 

for container in containers: 
    rating = container.div.div.div.span["class"] 

    comment_container = container.p 
    comment = comment_container[0] 

    print("rating" + rating) 
    print("comment" + comment) 

這是我的代碼的輸出：

Traceback (most recent call last): 
    File "trip_advisor.py", line 18, in <module> 
    comment = comment_container[0] 
    File "/anaconda/lib/python3.6/site-packages/bs4/element.py", line 1011, in 
__getitem__ 
    return self.attrs[key] 
KeyError: 0 

誰能幫我解決這個問題？謝謝。

Answer 1

您將無法通過在<class bs4.element.Tag'>使用索引來訪問內容，則需要.contents爲：

>>> container.p 
<p class="partial_entry">I was there couple of weeks ago on the weekend. There was an event but it was not very crowded thought and I actually like it. What drawn my attention is the PUB on the roof top. A Pub in a department store sound pretty...<span class="taLnk ulBlueLinks" onclick="ta.prwidgets.call('handlers.clickExpand',event,this);">More</span></p> 
>>> container.p.contents[0] 
'I was there couple of weeks ago on the weekend. There was an event but it was not very crowded thought and I actually like it. What drawn my attention is the PUB on the roof top. A Pub in a department store sound pretty...' 

除了這個問題，我不知道，如果你的rating刮真的得到你想要的東西，但是這解決了主要的錯誤：

for container in containers: 
    rating = container.div.div.div.span["class"] 
    comment_container = container.p.contents 
    comment = comment_container[0] 
    print("Rating: ", rating) 
    print("Comment: " + comment) 

此打印：

Rating: ['ui_bubble_rating', 'bubble_40'] 
Comment: I was there couple of weeks ago on the weekend. There was an event but it was not very crowded thought and I actually like it. What drawn my attention is the PUB on the roof top. A Pub in a department store sound pretty... 
Rating: ['ui_bubble_rating', 'bubble_50'] 
Comment: Show dc is very fascinating place that you must to go. The mega complex is very special from the others mall in thailand. I think you can touching and feeling of the hapiness. I went there few days ago for find some dining and spending...