使用BeautifulSoup刮表

Question

我有一個問題，我懷疑是相當直接。我有以下類型的頁面，從中我想收集在最後一個表中的信息（如果你向下滾動所有的方式，它是一個在標有「程序」框）：

http://www.europarl.europa.eu/sides/getDoc.do?type=REPORT&mode=XML&reference=A7-2010-2&language=EN

的HTML因爲我想刮這樣的長相表：

<tbody><tr class="doc_title"> 
<td style="background-image: url(&quot;/img/struct/navigation/gradient_blue.gif&quot;);" align="left" valign="top"><img src="/img/struct/functional/arrow_title_doc.gif" alt="" align="absmiddle" border="0" height="14" width="8"> <span style="font-weight: bold;">PROCEDURE</span></td><td style="background-image: url(&quot;/img/struct/navigation/gradient_blue.gif&quot;);" align="right" valign="top"> 
<table border="0" cellpadding="3" cellspacing="0" width="50"> 
<tbody><tr><td align="center"><a href="#top"><img src="/img/struct/functional/top_doc.gif" alt="" border="0" height="16" width="16"></a></td><td align="center"><img src="/img/struct/navigation/spacer.gif" alt="" border="0" height="10" width="15"></td><td align="center"><a href="#title2"><img src="/img/struct/functional/sort_up.gif" alt="" border="0" height="10" width="15"></a></td></tr></tbody></table></td></tr> 

<tr class="contents" valign="top"><td colspan="2"> 
<p></p><table style="border-collapse: collapse; width: 481.85pt;" align="center" cellspacing="0"> 
<tbody><tr style=""> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 38.24%;" rowspan="1" colspan="1"> 
<p style=""><span style="font-weight: bold;">Title</span></p> 
</td> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 61.76%;" rowspan="1" colspan="7"> 
<p style="">Mutual assistance for the recovery of claims relating to taxes, duties and other measures</p> 
</td> 
<td style="" rowspan="1" colspan="1"></td></tr> 

<tr style=""> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 38.24%;" rowspan="1" colspan="1"> 
<p style=""><span style="font-weight: bold;">References</span></p> 
</td> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 61.76%;" rowspan="1" colspan="7"> 
<p style=""><a href="http://ec.europa.eu/prelex/liste_resultats.cfm?CL=en&amp;ReqId=0&amp;DocType=COM&amp;DocYear=2009&amp;DocNum=0028">COM(2009)0028</a> – C6-0061/2009 – <a href="/oeil/FindByProcnum.do?lang=en&amp;procnum=CNS/2009/0007">2009/0007(CNS)</a></p> 
</td> 
<td style="" rowspan="1" colspan="1"></td></tr> 

<tr style=""> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 38.24%;" rowspan="1" colspan="1"> 
<p style=""><span style="font-weight: bold;">Date of consulting Parliament</span></p> 
</td> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 61.76%;" rowspan="1" colspan="7"> 
<p style="">16.2.2009</p> 
</td> 
<td style="" rowspan="1" colspan="1"></td></tr> 

<tr style=""> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 38.24%;" rowspan="1" colspan="1"> 
<p style=""><span style="font-weight: bold;">Committee responsible</span></p> 

<p style="">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Date announced in plenary</p> 
</td> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 61.76%;" rowspan="1" colspan="7"> 
<p style="">ECON</p> 

<p style="">19.10.2009</p> 
</td> 
<td style="" rowspan="1" colspan="1"></td></tr> 

<tr style=""> 
<td style="border-width: 0.75pt 1pt 0pt 0.75pt; border-style: solid solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 38.24%;" rowspan="1" colspan="1"> 
<p style=""><span style="font-weight: bold;">Committee(s) asked for opinion(s)</span></p> 

<p style="">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Date announced in plenary</p> 
</td> 
<td style="border-width: 1pt 0pt 0pt; border-style: solid none none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.88%;" rowspan="1" colspan="2"> 
<p style="">CONT</p> 

<p style="">19.10.2009</p> 
</td> 
<td style="border-width: 1pt 0pt 0pt; border-style: solid none none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.3%;" rowspan="1" colspan="2"> 
<p style="">JURI</p> 

<p style="">19.10.2009</p> 
</td> 
<td style="border-width: 1pt 0pt 0pt; border-style: solid none none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.3%;" rowspan="1" colspan="2"> 
<p style="">&nbsp;</p> 
</td> 
<td style="border-width: 0.75pt 1pt 0pt 0pt; border-style: solid solid none none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.3%;" rowspan="1" colspan="1"> 
<p style="">&nbsp;</p> 
</td> 
<td style="" rowspan="1" colspan="1"></td></tr> 

<tr style=""> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 38.24%;" rowspan="1" colspan="1"> 
<p style=""><span style="font-weight: bold;">Not delivering opinions</span></p> 

<p style="">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Date of decision</p> 
</td> 
<td style="border-width: 0.75pt 0pt 1pt; border-style: solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.88%;" rowspan="1" colspan="2"> 
<p style="">CONT</p> 

<p style="">1.10.2009</p> 
</td> 
<td style="border-width: 0.75pt 0pt 1pt; border-style: solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.3%;" rowspan="1" colspan="2"> 
<p style="">JURI</p> 

<p style="">5.10.2009</p> 
</td> 
<td style="border-width: 0.75pt 0pt 1pt; border-style: solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.3%;" rowspan="1" colspan="2"> 
<p style="">&nbsp;</p> 
</td> 
<td style="border-width: 0.75pt 1pt 0.75pt 0pt; border-style: solid solid solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.3%;" rowspan="1" colspan="1"> 
<p style="">&nbsp;</p> 
</td> 
<td style="" rowspan="1" colspan="1"></td></tr> 

<tr style=""> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 38.24%;" rowspan="1" colspan="1"> 
<p style=""><span style="font-weight: bold;">Rapporteur(s)</span></p> 

<p style="">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Date appointed</p> 
</td> 
<td style="border-width: 0.75pt 0pt 1pt; border-style: solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 20.59%;" rowspan="1" colspan="3"> 
<p style="">Theodor Dumitru Stolojan</p> 

<p style="">21.7.2009</p> 
</td> 
<td style="border-width: 0.75pt 0pt 1pt; border-style: solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 20.59%;" rowspan="1" colspan="2"> 
<p style="">&nbsp;</p> 
</td> 
<td style="border-width: 0.75pt 1pt 0.75pt 0pt; border-style: solid solid solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 20.59%;" rowspan="1" colspan="2"> 
<p style="">&nbsp;</p> 
</td> 
<td style="" rowspan="1" colspan="1"></td></tr> 

<tr style=""> 
<td style="border-width: 0.75pt 1pt 0pt 0.75pt; border-style: solid solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 38.24%;" rowspan="1" colspan="1"> 
<p style=""><span style="font-weight: bold;">Discussed in committee</span></p> 
</td> 
<td style="border-width: 1pt 0pt 0pt; border-style: solid none none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.88%;" rowspan="1" colspan="2"> 
<p style="">10.11.2009</p> 
</td> 
<td style="border-width: 1pt 0pt 0pt; border-style: solid none none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.3%;" rowspan="1" colspan="2"> 
<p style="">1.12.2009</p> 
</td> 
<td style="border-width: 1pt 0pt 0pt; border-style: solid none none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.3%;" rowspan="1" colspan="2"> 
<p style="">21.1.2010</p> 
</td> 
<td style="border-width: 0.75pt 1pt 0pt 0pt; border-style: solid solid none none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.3%;" rowspan="1" colspan="1"> 
<p style="">&nbsp;</p> 
</td> 
<td style="" rowspan="1" colspan="1"></td></tr> 

<tr style=""> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 38.24%;" rowspan="1" colspan="1"> 
<p style=""><span style="font-weight: bold;">Date adopted</span></p> 
</td> 
<td style="border-width: 0.75pt 0pt 1pt; border-style: solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.88%;" rowspan="1" colspan="2"> 
<p style="">27.1.2010</p> 
</td> 
<td style="border-width: 0.75pt 0pt 1pt; border-style: solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.3%;" rowspan="1" colspan="2"> 
<p style="">&nbsp;</p> 
</td> 
<td style="border-width: 0.75pt 0pt 1pt; border-style: solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.3%;" rowspan="1" colspan="2"> 
<p style="">&nbsp;</p> 
</td> 
<td style="border-width: 0.75pt 1pt 0.75pt 0pt; border-style: solid solid solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 15.3%;" rowspan="1" colspan="1"> 
<p style="">&nbsp;</p> 
</td> 
<td style="" rowspan="1" colspan="1"></td></tr> 

<tr style=""> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 38.24%;" rowspan="1" colspan="1"> 
<p style=""><span style="font-weight: bold;">Result of final vote</span></p> 
</td> 
<td style="border-width: 0.75pt 0pt 1pt; border-style: solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 12.94%;" rowspan="1" colspan="1"> 
<p style="">+:</p> 

<p style="">–:</p> 

<p style="">0:</p> 
</td> 
<td style="border-width: 0.75pt 1pt 0.75pt 0pt; border-style: solid solid solid none; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 48.82%;" rowspan="1" colspan="6"> 
<p style="">39</p> 

<p style="">0</p> 

<p style="">1</p> 
</td> 
<td style="" rowspan="1" colspan="1"></td></tr> 

<tr style=""> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 38.24%;" rowspan="1" colspan="1"> 
<p style=""><span style="font-weight: bold;">Members present for the final vote</span></p> 
</td> 
<td style="border-width: 0.75pt 1pt 0.75pt 0.75pt; border-style: solid; border-color: rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 61.76%;" rowspan="1" colspan="7"> 
<p style="">Burkhard Balz, Sharon Bowles, Udo Bullmann, Pascal Canfin, Nikolaos Chountis, George Sabin Cutaş, Leonardo Domenici, Derk Jan Eppink, Markus Ferber, Elisa Ferreira, Vicky Ford, José Manuel García-Margallo y Marfil, Jean-Paul Gauzès, Sylvie Goulard, Enikő Győri, Liem Hoang Ngoc, Eva Joly, Othmar Karas, Wolf Klinz, Jürgen Klute, Werner Langen, Astrid Lulling, Arlene McCarthy, Ivari Padar, Alfredo Pallone, Anni Podimata, Antolín Sánchez Presedo, Olle Schmidt, Edward Scicluna, Peter Simon, Peter Skinner, Theodor Dumitru Stolojan, Ivo Strejček, Kay Swinburne, Marianne Thyssen, Ramon Tremosa i Balcells</p> 
</td> 
<td style="" rowspan="1" colspan="1"></td></tr> 

<tr style=""> 
<td style="border-left: 0.75pt solid rgb(0, 0, 0); border-right: 1pt solid rgb(0, 0, 0); border-top: 0.75pt solid rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 38.24%;" rowspan="1" colspan="1"> 
<p style=""><span style="font-weight: bold;">Substitute(s) present for the final vote</span></p> 
</td> 
<td style="border-left: 0.75pt solid rgb(0, 0, 0); border-right: 1pt solid rgb(0, 0, 0); border-top: 0.75pt solid rgb(0, 0, 0); padding: 2.8pt 5.1pt; vertical-align: top; width: 61.76%;" rowspan="1" colspan="7"> 
<p style="">Marta Andreasen, Sophie Briard Auconie, David Casa, Danuta Jazłowiecka, Arturs Krišjānis Kariņš, Philippe Lamberts, Andreas Schwab</p> 
</td> 
<td style="" rowspan="1" colspan="1"></td></tr> 

<tr style=""> 
<td style="border: medium none; margin: 0pt; padding: 0pt; width: 38.24%;" rowspan="1" colspan="1"></td> 
<td style="border: medium none; margin: 0pt; padding: 0pt; width: 12.94%;" rowspan="1" colspan="1"></td> 
<td style="border: medium none; margin: 0pt; padding: 0pt; width: 2.94%;" rowspan="1" colspan="1"></td> 
<td style="border: medium none; margin: 0pt; padding: 0pt; width: 4.71%;" rowspan="1" colspan="1"></td> 
<td style="border: medium none; margin: 0pt; padding: 0pt; width: 10.58%;" rowspan="1" colspan="1"></td> 
<td style="border: medium none; margin: 0pt; padding: 0pt; width: 10%;" rowspan="1" colspan="1"></td> 
<td style="border: medium none; margin: 0pt; padding: 0pt; width: 5.29%;" rowspan="1" colspan="1"></td> 
<td style="border: medium none; margin: 0pt; padding: 0pt; width: 15.3%;" rowspan="1" colspan="1"></td> 
<td style="" rowspan="1" colspan="1"></td></tr> 
</tbody></table> 
</td></tr> 
</tbody> 

我現在面臨的問題是，對錶標籤不具有標識符（據我可以告訴），所以我不知道如何選擇這個表並從中颳去信息。我一直在使用BeautifilSoup來獲取網站上的其他信息，但我對如何刮這張桌子感到不知所措。

如果有人能告訴我如何繼續我將不勝感激！

隨着親切的問候，

托馬斯

Answer 1

如果你是一個聰明一點，您可以找到其他屬性的元素。我抓住了這個鏡頭來抓取你的數據，它可能不是最好的–，但它讓你接近。

我注意到的第一件事情是，在第二次出現「PROCEDURE」（第一個是鏈接，第二個是標題）後，您肯定需要數據。所以，我拆對：

data = html.split("PROCEDURE", 2)[2] 

於是，我找了<td>標籤與rowspan=1：

bs = BeautifulSoup.BeautifulSoup(data) 
tds = bs.findAll("td", { "rowspan": 1 }) 

越來越近......

>>> tds[0].text 
u'Title' 
>>> tds[1].text 
u'Mutual assistance for the recovery of claims relating to taxes, duties and other measures' 
>>> tds[3].text 
u'References' 
>>> tds[4].text 
u'COM(2009)00282009/0007(CNS)2009 a>' 

請注意，我在跳過指數2tds，因爲他們使用間隔或其他東西（它是空的）。無論如何，這是一個開始。我在BeautifulSoup上找到的真正訣竅就是隻將它提供給你所知道的那個區域的數據，因爲那樣的話就不那麼容易了。它也自on接受不好看的輸入，所以不要害怕餵它垃圾。

我在元素列表中走得更遠一些，它並不完美。您需要細化搜索，因爲它們的值在<td>之間有<td>個元素。