0
尋求幫助,以循環訪問網站上的所有選項卡以捕獲所有相關信息。Python Web Scrape Cycle選項卡
在以下站點中,有幾個標籤分別標記爲5x5,5x10x5,10x10等。我不確定如何構造它,以便它會通過選項卡並在我的腳本中編寫循環。感謝您的幫助。
下面是python腳本;
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
import csv
urls = [
'https://www.lifestorage.com/storage-units/florida/orlando/32810/610-near-lockhart/?size=5x5'
]
filename = 'life_storage.csv'
f = open(filename, 'a+')
csv_writer = csv.writer(f)
headers = ['unit_size', 'unit_type', 'description', 'online_price', 'reg_price', 'store_address', 'store_city', 'store_state', 'store_postalcode' ]
##unit_size = 5'x10' withouth the '
##unit_type = climate controlled or not (this could be blank if non-climate)
##descirption = the level it's on and type of access.
##online_price = $##/mo text
##reg_price = the scratched off $## text
csv_writer.writerow(headers)
for my_url in urls:
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
page_soup = soup(page_html, 'html.parser')
store_locator = page_soup.findAll("div", {"itemprop": "address"})
containers = page_soup.findAll("ul", {"id": "spaceList"})
for container in containers:
for store_location in store_locator:
store_address1 = store_location.find("span", {"itemprop": "streetAddress"})
store_address = store_address1.text
store_city1 = store_location.find("span", {"itemprop": "addressLocality"})
store_city = store_city1.text
store_state1 = store_location.find("span", {"itemprop": "addressRegion"})
store_state = store_state1.text
store_postalcode1 = store_location.find("span", {"itemprop": "postalCode"})
store_postalcode = store_postalcode1.text
title_container = container.find("div", {"class": "storesRow"})
unit_size = title_container.text
unit_container = container.find("div", {"class": "storesRow"})
unit_type = unit_container.strong.text
description_container = container.find("ul", {"class": "features"})
description = description_container.text
online_price_container = container.find("div", {"class": "priceBox"})
online_price = online_price_container.strong.text
reg_price_container = container.find("div", {"class": "priceBox"})
reg_price = reg_price_container.i.text
csv_writer.writerow([unit_size, unit_type, description, online_price, reg_price, store_address, store_city, store_state, store_postalcode])
f.close()
下面是與循環相關的html正文的片段;
//////////\\\\\\\Description BOX
<div class="storesRow">
<strong>
<a href="/reservation/choose/?store=610&type=1"> 5' x 5'<sup>*</sup> - Climate Controlled </a>
</strong>
<ul class="features">
<li>Indoor access</li>
<li>Ground Level</li>
</ul>
</div>
//////////\\\\\\\\\PRICE BOX
<div class="priceBox">
<strong>
$25/mo
<i> $27</i>
</strong>
<em class="pOnly ">Phone & online only</em>
<div class="specialsMessage">
</div>
</div>
//////////\\\\\\\\\ADDRESS BOX
<div itemprop="address" itemscope="" itemtype="https://schema.org/PostalAddress">
<em>
<i class="fa fa-map-marker"></i>
<span itemprop="streetAddress">7244 Overland Rd </span>
<span itemprop="addressLocality">Orlando</span>,
<span itemprop="addressRegion">FL</span>
<span itemprop="postalCode">32810</span>
</em>
</div>
你有什麼期望的輸出? –
@ Paul - 我編輯了目前的輸出結果和我想要的結果。這將遍歷所有選項卡,因此對於每個新的大小,它將被放在適當標題下的新行中。 –
你有錯誤的注意 - '''writerow()'應該在'for'內部 - 然後它應該把所有項目添加到你的列表中。 – furas