使用BeautifulSoup從HTML中抽取具有特定字符串的表格

Question

我試圖使用BeautifulSoup和Python中的請求來提取播放數據，但是此代碼只是爲數組「表格」返回一個空數組[]。我對這些圖書館比較陌生，但是我在使用類似網站（即來自其他大學遊戲的其他逐幀播放數據）執行類似任務時使用了類似的語法。我感興趣的提取文本包含與「頂第1局的」開始的表格中，「第2局」等的底部澄清，如果有不清楚的地方，謝謝，請發表評論！

from bs4 import BeautifulSoup 

import requests 

header = {'User-agent' : 'Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5'} 

url = requests.get("http://www.belmontbruins.com/sports/m-basebl/2016-17/boxscores/20170407_c6td.xml?view=plays", headers = header).text 

soup = BeautifulSoup(url, 'html.parser') 

with open('test.txt','w+') as myfile: 
    table = soup.find_all('table', text = ['Top', 'Bottom']) 
    print(table) 
    for eachtable in table: 
     rows = eachtable.find_all('tr') 
     for tr in rows: 
      cols = tr.find_all('td') 
      for td in cols: 
       myfile.write(td.text + '\n') 

Answer 1

當您搜索Top|Bottom時，它會在HTML樹中找到文本節點。您可以使用瀏覽器查看該頁面，並且可以看到結構如下所示：table > caption > h3 > "Top of ..." 因此，在找到給定的文本節點後，您必須使用element.parent.parent.parent獲取3個級別，以獲取包含文本節點的表格。

這裏是完整的代碼：

from bs4 import BeautifulSoup 

import re 
import requests 

header = {'User-agent' : 'Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5'} 

url = requests.get("http://www.belmontbruins.com/sports/m-basebl/2016-17/boxscores/20170407_c6td.xml?view=plays", headers = header).text 

soup = BeautifulSoup(url, 'html.parser') 

elements = soup(text=re.compile('Top|Bottom')) 
with open('test.txt','w+') as myfile: 
    for element in elements: 
     rows = element.parent.parent.parent.find_all('tr') 
     for tr in rows: 
      cols = tr.find_all('td') 
      for td in cols: 
       myfile.write(td.text + '\n') 

Answer 2

import requests 
header = {'User-agent' : 'Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5'} 

url = requests.get("http://www.belmontbruins.com/sports/m-basebl/2016-17/boxscores/20170407_c6td.xml?view=plays", headers = header).text 

soup = BeautifulSoup(url, 'html.parser') 

table = soup.findAll('table',{'class':'striped'}) 

thefile = open('test.txt', 'w') 
for i in table: 
    for j in i.findAll('td',{'class':'text'}): 
     txt = str(j.get_text()) 
     if txt.startswith('Top of 1st Inning') or txt.startswith('Bottom of 2nd Inning'): 
       thefile.write("%s\n" % item)