將網站中的數據提取到python中的字典中

以下是從Indeed.com提取特定作業數據的代碼和相應輸出。隨着數據我有很多垃圾，我想分開標題，位置，工作描述和其他重要功能。我怎樣才能將它轉換成字典？將網站中的數據提取到python中的字典中

from bs4 import BeautifulSoup 
import urllib2 
final_site = 'http://www.indeed.com/cmp/Pullskill-techonoligies/jobs/Data-Scientist-229a6b09c5eb6b44?q=%22data+scientist%22' 
html = urllib2.urlopen(final_site).read() 
soup = BeautifulSoup(html) 
deep = soup.find("td","snip") 
deep.get("p","ul") 
deep.get_text(strip= True)

輸出：

u'Title : Data ScientistLocation : Seattle WADuration : Fulltime/PermanentJob Responsibilities:Implement advanced and predictive analytics models usingJava,R, and Pythonetc.Develop deep expertise with Company\u2019s data warehouse, systems, product and other resources.Extract, collate and analyze data from a variety of sources to provide insights to customersCollaborate with the research team to incorporate qualitative insights into projects where appropriateKnowledge, Skills and Experience:Exceptional problem solving skillsExperience withJava,R, and PythonAdvanced data mining and predictive modeling (especially Machine learning techniques) skillsMust have statistics orientation (Theory and applied)3+ years of business experience in an advanced analytics roleStrong Python and R programming skills are required. SAS, MATLAB will be plusStrong SQL skills are looked for.Analytical and decisive strategic thinker, flexible problem solver, great team player;Able to effectively communicate to all levelsImpeccable attention to detail and very strong ability to convert complex data into insights and action planThanksNick ArthurLead Recruiternick(at)pullskill(dot)com201-497-1010 Ext: 106Salary: $120,000.00 /yearRequired experience:Java And Python And R And PHD Level Education: 4 years5 days ago-save jobwindow[\'result_229a6b09c5eb6b44\'] = {"showSource": false, "source": "Indeed", "loggedIn": false, "showMyJobsLinks": true,"undoAction": "unsave","relativeJobAge": "5 days ago","jobKey": "229a6b09c5eb6b44", "myIndeedAvailable": true, "tellAFriendEnabled": false, "showMoreActionsLink": false, "resultNumber": 0, "jobStateChangedToSaved": false, "searchState": "", "basicPermaLink": "http://www.indeed.com", "saveJobFailed": false, "removeJobFailed": false, "requestPending": false, "notesEnabled": true, "currentPage" : "viewjob", "sponsored" : false, "reportJobButtonEnabled": false};\xbbApply NowPlease review all application instructions before applying to Pullskill Technologies.(function(d, s, id){var js, iajs = d.getElementsByTagName(s)[0], iaqs = \'vjtk=1aa24enhqagvcdj7&hl=en_US&co=US\'; if (d.getElementById(id)){return;}js = d.createElement(s); js.id = id; js.async = true; js.src = \'https://apply.indeed.com/indeedapply/static/scripts/app/bootstrap.js\'; js.setAttribute(\'data-indeed-apply-qs\', iaqs); iajs.parentNode.insertBefore(js, iajs);}(document, \'script\', \'indeed-apply-js\'));Recommended JobsData Scientist, Energy AnalyticsRenew Financial-Oakland, CARenew Financial-5 days agoData ScientistePrize-Seattle, WAePrize-7 days agoData ScientistDocuSign-Seattle, WADocuSign-12 days agoEasily applyEngineer - US Citizen or Permanent ResidentVoxel Innovations-Raleigh, NCIndeed-8 days agoEasily applyData ScientistUnity Technologies-San Francisco, CAUnity Technologies-22 days agoEasily apply'

來源

2016-01-27 dsl1990

找到工作摘要元素，發現裏面所有b元素，並用:分裂每個b元素的文本：

for elm in soup.find("span", id="job_summary").p.find_all("b"): 
    label, text = elm.get_text().split(" : ") 

    print(label.strip(), text.strip())

來源

2016-01-27 18:36:58 alecxe

感謝。這只是給標題，位置和位置。我希望在工作中獲得工作職責和其他信息。無論出現在工作崗位上的任何信息都是有價值的。 – dsl1990

-1

如果您的輸出始終具有相同的結構，你可以使用正則表達式來創建字典。

dict = {} 
title_match = re.match(r'Title : (.+)(?=Location)', output) 
dict['Title'] = title_match.group(1) 
location_match = re.match(r'Location : (.+)(?=Duration)', output) 
dict['Location'] = location_match.group(1)

當然，這是一個非常脆弱的解決方案，它可能會更好地爲您使用BeautifulSoup的內置解析得到你想要的結果，我想他們可能是由標準的標籤包圍。

來源

2016-01-27 18:33:52 Barnabus

將網站中的數據提取到python中的字典中

回答

相關問題