Python：維護會話以便登錄後訪問所有頁面

Question

我正在使用python來解析一些網頁。我用漂亮的腳本來分析一個登錄頁面和發送有效載荷登錄：

import bs4 as bs 
import urllib.request 
import requests 
import webbrowser 
import urllib.parse 

primary_url = "SOME URL YOU WANT TO LOG IN" 
sauce = urllib.request.urlopen(primary_url).read() 
soup = bs.BeautifulSoup(sauce,"html.parser") 

for form_list in soup.find_all('form'): 

    action_value = form_list.get('action') 
    action_url = urllib.parse.urljoin(primary_url, action_value) 
    method_value = form_list.get('method') 

    if (method_value == "post"): 
     payload = dict() 
     inputs1 = form_list.findAll('input',type ="email") 
     for i in inputs1: 
      input2 = i.get('name') 
      script_value = 'USER-EMAIL' 
      payload[input2] = script_value 
     inputs2 = form_list.findAll('input',type ="password")  
     for j in inputs2: 
      input3 = j.get('name') 
      script_value = 'PASSWORD-FOR-EMAIL' 
      payload[input3] = script_value   

     r = requests.post(action_url, payload) 
     with open("result_page.html", "w") as f: 
      f.write(str(r.content)) 
     sessionid = r.cookies.get('SESSIONID') #ERROR 
     print(sessionid) #ERROR 

# CODE TO ACCESS FURTHER PAGES . . . 

但是我不能夠再訪問頁面，因爲我需要登錄才能訪問這些頁面。

Answer 1

您可以重新編寫代碼以使用Python請求高效地解決您陳述的問題。

username = "Your Username" 
password = "Password" 
sauce = requests.get(url, auth=(username, password)) 
sauce = sauce.content 
soup = bs.BeautifulSoup(sauce,"html.parser") 
print(soup) 

您不需要'維護'會話以訪問其他頁面。您只需發送auth = {username, password}到您想要解析的每個頁面。

Answer 2

你可以用requests.Session()來做到這一點。創建一個session，登錄並解析。登錄後，您也可以解析其他頁面。
您可以閱讀文檔here。

import bs4 as bs 
import requests 
import urllib.parse 

primary_url = "SOME URL YOU WANT TO LOG IN" 
sess = requests.Session() 
sauce = sess.get(primary_url) 
soup = bs.BeautifulSoup(sauce.text,"html.parser") 

for form_list in soup.find_all('form'): 

    action_value = form_list.get('action') 
    action_url = urllib.parse.urljoin(primary_url, action_value) 
    method_value = form_list.get('method') 

    if (method_value == "post"): 
     payload = dict() 

     inputs1 = form_list.find_all('input',type ="email") 
     for i in inputs1: 
      input2 = i.get('name') 
      script_value = 'USER-EMAIL' 
      payload[input2] = script_value 

     inputs2 = form_list.find_all('input',type ="password")  
     for j in inputs2: 
      input3 = j.get('name') 
      script_value = 'PASSWORD-FOR-EMAIL' 
      payload[input3] = script_value   

     r = sess.post(action_url, data=payload) 
     with open("result_page.html", "w") as f: 
      f.write(str(r.content)) 

編輯
登陸，你需要做一個post請求。在窗體中找到username，password字段的名稱，然後創建一個字典併發送提出post請求的數據。