在python中將字典轉換爲數據框

Question

我的循環的第一件事是不按我想要的方式工作。它逐步爲特定網站提供字典內的鏈接。我想填補它在once.My輸出是這樣的：

{'Banks – Assets': {'link': 'https://data.gov.au/dataset/banks-assets'}, 'Consolidated Exposures – Immediate and Ultimate Risk Basis': {}, 'Foreign Exchange Transactions and Holdings of Official Reserve Assets': {}, 'Finance Companies and General Financiers – Selected Assets and Liabilities': {}, 'Liabilities and Assets – Monthly': {}, 'Consolidated Exposures – Immediate Risk Basis – International Claims by Country': {}, 'Consolidated Exposures – Ultimate Risk Basis': {}, 'Banks – Consolidated Group off-balance Sheet Business': {}, 'Liabilities of Australian-located Operations': {}, 'Building Societies – Selected Assets and Liabilities': {}, 'Consolidated Exposures – Immediate Risk Basis – Foreign Claims by Country': {}, 'Banks – Consolidated Group Impaired Assets': {}, 'Assets and Liabilities of Australian-Located Operations': {}, 'Managed Funds': {}, 'Daily Net Foreign Exchange Transactions': {}, 'Consolidated Exposures-Immediate Risk Basis': {}, 'Public Unit Trust': {}, 'Securitisation Vehicles': {}, 'Assets of Australian-located Operations': {}, 'Banks – Consolidated Group Capital': {}} 
{'Banks – Assets': {'link': 'https://data.gov.au/dataset/banks-assets'}, 'Consolidated Exposures – Immediate and Ultimate Risk Basis': {'link': 'https://data.gov.au/dataset/consolidated-exposures-immediate-and-ultimate-risk-basis'}, 'Foreign Exchange Transactions and Holdings of Official Reserve Assets': {}, 'Finance Companies and General Financiers – Selected Assets and Liabilities': {}, 'Liabilities and Assets – Monthly': {}, 'Consolidated Exposures – Immediate Risk Basis – International Claims by Country': {}, 'Consolidated Exposures – Ultimate Risk Basis': {}, 'Banks – Consolidated Group off-balance Sheet Business': {}, 'Liabilities of Australian-located Operations': {}, 'Building Societies – Selected Assets and Liabilities': {}, 'Consolidated Exposures – Immediate Risk Basis – Foreign Claims by Country': {}, 'Banks – Consolidated Group Impaired Assets': {}, 'Assets and Liabilities of Australian-Located Operations': {}, 'Managed Funds': {}, 'Daily Net Foreign Exchange Transactions': {}, 'Consolidated Exposures-Immediate Risk Basis': {}, 'Public Unit Trust': {}, 'Securitisation Vehicles': {}, 'Assets of Australian-located Operations': {}, 'Banks – Consolidated Group Capital': {}} 

第二件事我想使數據幀，它像：

Titles             Links 
Banks - Assets      https://data.gov.au/dataset/banks-assets 
Consolidated Exposures – Immediate and Ultimate Risk Basis https://data.gov.au/dataset/consolidated-exposures-immediate-and-ultimate-risk-basis 

等等... 我的代碼：

webpage4_urls = ["https://data.gov.au/dataset?q=&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&groups=sciences&organization=departmentofagriculturefisheriesandforestry&_groups_limit=0", 
       "https://data.gov.au/dataset?q=&organization=commonwealthscientificandindustrialresearchorganisation&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&groups=sciences&_groups_limit=0", 
       "https://data.gov.au/dataset?q=&organization=bureauofmeteorology&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&groups=sciences&_groups_limit=0", 
       "https://data.gov.au/dataset?q=&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&groups=sciences&organization=tasmanianmuseumandartgallery&_groups_limit=0", 
       "https://data.gov.au/dataset?q=&organization=department-of-industry&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&groups=sciences&_groups_limit=0"] 
for i in webpage4_urls: 
    wiki2 = i 
    page= urllib.request.urlopen(wiki2) 
    soup = BeautifulSoup(page) 

    lobbying = {} 
    data2 = soup.find_all('h3', class_="dataset-heading") 
    for element in data2: 
     lobbying[element.a.get_text()] = {} 
    data2[0].a["href"] 
    prefix = "https://data.gov.au" 
    for element in data2: 
     lobbying[element.a.get_text()]["link"] = prefix + element.a["href"] 
     print(lobbying) 

Answer 1

我想你需要DataFrame.from_dict + DataFrame.rename_axis + DataFrame.reset_index：

for element in data2: 
    lobbying[element.a.get_text()]["link"] = prefix + element.a["href"] 
    #print(lobbying) 
    df = pd.DataFrame.from_dict(lobbying, orient='index').rename_axis('Titles').reset_index() 
    print (df) 

編輯：

dfs = [] 
for i in webpage4_urls: 
    wiki2 = i 
    page= urllib.request.urlopen(wiki2) 
    soup = BeautifulSoup(page) 

    lobbying = {} 
    data2 = soup.find_all('h3', class_="dataset-heading") 
    for element in data2: 
     lobbying[element.a.get_text()] = {} 
    data2[0].a["href"] 
    prefix = "https://data.gov.au" 
    for element in data2: 
     print() 
     lobbying[element.a.get_text()]["link"] = prefix + element.a["href"] 
     #print(lobbying) 
     df = pd.DataFrame.from_dict(lobbying, orient='index').rename_axis('Titles').reset_index() 
     dfs.append(df) 

df = pd.concat(dfs, ignore_index=True) 
print (df) 
df.to_csv('output.csv')