從使用Python中的beautifulsoup的網站中提取數字

Question

我正在嘗試使用urllib來抓取html頁面，然後使用beautifulsoup來提取數據。我想從comments_42.html中獲取所有數字並打印出它們的總和，然後顯示數據的數量。 這是我的代碼，我正在嘗試使用正則表達式，但它不適用於我。

import urllib 
from bs4 import BeautifulSoup 
url = 'http://python-data.dr-chuck.net/comments_42.html' 
html = urllib.urlopen(url).read() 
soup = BeautifulSoup(html,"html.parser") 
tags = soup('span') 
for tag in tags: 
    print tag 

Answer 1

使用BeautifulSoup的findAll（）方法提取所有帶有'comments'類的span標籤，因爲它們包含了您需要的信息。然後您可以根據您的要求對它們執行任何操作。

soup = BeautifulSoup(html,"html.parser") 
data = soup.findAll("span", { "class":"comments" }) 
numbers = [d.text for d in data] 

這裏是輸出：

[u'100', u'97', u'87', u'86', u'86', u'78', u'75', u'74', u'72', u'72', u'72', u'70', u'70', u'66', u'66', u'65', u'65', u'63', u'61', u'60', u'60', u'59', u'59', u'57', u'56', u'54', u'52', u'52', u'51', u'47', u'47', u'41', u'41', u'41', u'38', u'35', u'32', u'31', u'24', u'19', u'19', u'18', u'17', u'16', u'13', u'8', u'7', u'1', u'1', u'1'] 

Answer 2

@學習者的解決方案是完全正確的！但如果你想要做更多的名稱和註釋，你可以做到這一點，它返回名稱和註釋的列表：

from BeautifulSoup import BeautifulSoup 
import re 
import urllib 
url = 'http://python-data.dr-chuck.net/comments_42.html' 
html = urllib.urlopen(url).read() 
soup = BeautifulSoup(html) 
all = soup.findAll('span',{'class':'comments'},text=re.compile(r'[0-9]{0,4}')) #use regex to extract only numbers 
cleaned = filter(lambda x: x!=u'\n',all)[4:] 
In [18]: cleaned 
Out[18]: 
[u'Leven', 
u'100', 
u'Mahdiya', 
u'97', 
u'Ajayraj', 
u'87', 
u'Lillian', 
u'86', 
u'Aon', 
u'86', 
u'Ruaraidh', 
u'78', 
u'Gursees', 
u'75', 
u'Emmanuel', 
u'74', 
u'Christy', 
u'72', 
u'Annoushka', 
u'72', 
u'Inara', 
u'72', 
u'Caite', 
u'70', 
u'Rosangel', 
u'70', 
u'Iana', 
u'66', 
u'Anise', 
u'66', 
u'Jaosha', 
u'65', 
u'Cadyn', 
u'65', 
u'Edward', 
u'63', 
u'Charlotte', 
u'61', 
u'Sammy', 
u'60', 
u'Zarran', 
u'60',.....] # 

Answer 3

不要忘記，你必須要想在代碼中使用它們導入正則表達式。

import re

Answer 4

我從Coursera開始學習同樣的課程。不要去尋求上述解決方案，你介意嘗試這一個。直到上述問題，我覺得這個問題屬於我們所瞭解的範圍。它絕對爲我工作。

import urllib 
import re 
from bs4 import * 

url = 'http://python-data.dr-chuck.net/comments_216543.html' 
html = urllib.urlopen(url).read() 

soup = BeautifulSoup(html,"html.parser") 
sum=0 
# Retrieve all of the anchor tags 
tags = soup('span') 
for tag in tags: 
    # Look at the parts of a tag 
    y=str(tag) 
    x= re.findall("[0-9]+",y) 
    for i in x: 
     i=int(i) 
     sum=sum+i 
print sum 

Answer 5

做它的基本途徑...

# Retrieve all of the anchor tags 
tags = soup('span') 
sum = 0 
count = 0 
for tag in tags: 
# Look at the parts of a tag 

    #print tag.contents[0] 
    num = float(tag.contents[0]) 
    #print num 
    sum = sum + num 
    count = count + 1 

print 'count:',count 
print 'sum:',sum  

Answer 6

我這樣做的光標，它給了我所有的正確答案。希望它幫助;）

from urllib.request import urlopen 
from bs4 import BeautifulSoup 
import ssl 

# Ignore SSL certificate errors 
ctx = ssl.create_default_context() 
ctx.check_hostname = False 
ctx.verify_mode = ssl.CERT_NONE 

url = input('Enter - ') 
html = urlopen(url, context=ctx).read() 
soup = BeautifulSoup(html,"html.parser") 

# Retrieve all of the anchor tags 
tags = soup('span') 
sum = 0 
count = 0 
for tag in tags: 
# Look at the parts of a tag 

    #print tag.contents[0] 
    num = float(tag.contents[0]) 
    #print num 
    sum = sum + num 
    count = count + 1 

print ('count:', count) 
print ('sum:', sum) 

Answer 7

import urllib.request,urllib.parse,urllib.error 

import re 

from bs4 import BeautifulSoup 

url = input('Enter - ') 


html = urllib.request.urlopen(url).read() 

soup = BeautifulSoup(html,"html.parser") 

tags=soup('span') 

sum=0 

for tag in tags: 

    x=re.findall("[0-9]+",tag) 



    for i in x: 

     z=int(i) 


     sum=sum+i 


print(sum)