2

我有這個網頁:http://waterdata.usgs.gov/nwis/wys_rpt?dd_parm_cds=002_00060&wys_water_yr=2015&site_no=06935965美麗的湯不會從HTML對象返回全表

,我是希望能颳去這樣的信息:

enter image description here

的信息存儲在這裏一個表,顯示標題後的第一個兩行展開:

enter image description here

因此,我設置並學習了一個BeautifulSoup,並找到我的表格(這是頁面上的最後一個,因此是tables[-1]),但它不會拾取整個表格 - 在「年度總計」行名稱/條目之後停止。

代碼:

from bs4 import BeautifulSoup 
import requests 


base_url = 'http://waterdata.usgs.gov/nwis/wys_rpt?dd_parm_cds=002_00060&wys_water_yr=2015&site_no=' 
site = '06935965' 

url = base_url + site 
r = requests.get(url) 

soup = BeautifulSoup(r.text,'html.parser') 
tables = soup.find_all('table') 

table = tables[-1] 

print(table.text) 

輸出:

SUMMARY STATISTICS 



Water Year 2015 
Water Years 2000 - 2015 




Annual total 

而這一切!整個表被拾取的請求調用:

<table class='tables'> 
<caption class='table_caption'>SUMMARY STATISTICS</caption> 
<thead class='thead'> 
<tr> 
<th class='tables_th'></th> 
<th class='tables_th' colspan='2'>Water Year 2015</th> 
<th class='tables_th' colspan='2'>Water Years 2000 - 2015</th> 
</tr> 
</thead> 
<tbody> 
<tr> 
<td class='tables_date'>Annual total</th> 
<td>41,170,000<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
<td><span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
</tr> 
<tr> 
<td class='tables_date'>Annual mean</th> 
<td>112,800<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
<td>87,520<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
</tr> 
<tr> 
<td class='tables_date'>Highest annual mean</th> 
<td><span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
<td>154,900<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td>2010</td> 
</tr> 
<tr> 
<td class='tables_date'>Lowest annual mean</th> 
<td><span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
<td>42,090<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td>2006</td> 
</tr> 
<tr> 
<td class='tables_date'>Highest daily mean</th> 
<td>342,000<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td>Jun 20</td> 
<td>398,000<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td>Jun 02, 2013</td> 
</tr> 
<tr> 
<td class='tables_date'>Lowest daily mean</th> 
<td>40,600<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td>Jan 19</td> 
<td>22,900<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td>Jan 26, 2003</td> 
</tr> 
<tr> 
<td class='tables_date'>Annual 7-day minimum</th> 
<td>41,410.0<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td>Jan 18</td> 
<td>23,630.0<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td>Jan 24, 2003</td> 
</tr> 
<tr> 
<td class='tables_date'>Maximum peak flow</th> 
<td>344,000<span class='padding'><sup>a</sup>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td>Jun 20</td> 
<td>409,000<span class='padding'><sup>a</sup>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td>Jun 02, 2013</td> 
</tr> 
<tr> 
<td class='tables_date'>Maximum peak stage</th> 
<td>31.76<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td>Jun 20</td> 
<td>33.80<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td>Jun 02, 2013</td> 
</tr> 
<tr> 
<td class='tables_date'>Annual runoff (cfsm)</th> 
<td>0.215<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
<td>0.165<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
</tr> 
<tr> 
<td class='tables_date'>Annual runoff (inches)</th> 
<td>2.92<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
<td>2.25<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
</tr> 
<tr> 
<td class='tables_date'>10 percent exceeds</th> 
<td>249,000<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
<td>173,000<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
</tr> 
<tr> 
<td class='tables_date'>50 percent exceeds</th> 
<td>76,800<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
<td>63,100<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
</tr> 
<tr> 
<td class='tables_date'>90 percent exceeds</th> 
<td>50,500<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
<td>35,700<span class='padding'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span></td> 
<td></td> 
</tr> 
</tbody> 
</table> 

有人能看到爲什麼美麗的湯被省略了表的其餘部分?

回答

2

你只需要change the parser到一個更寬鬆:

soup = BeautifulSoup(r.text, 'html5lib') 

lxml會處理這種情況下:

soup = BeautifulSoup(r.text, 'lxml') 
+0

'html5lib' 漂亮的工作。 – traggatmot