如何使用PHP提交表單並從其他網站檢索文件？

Question

我需要每天從我的本地服務器下載http://www.carriersoftwaredata.com/Login.aspx?file=FHWA的CSV文件。我的計劃是執行一個運行php腳本的cron作業來執行此操作。但是，在允許下載文件之前，該頁面需要輸入用戶名和密碼。如果我在我的Web瀏覽器中訪問此頁面，請輸入用戶名和密碼，然後提交表單，然後我的瀏覽器會顯示一個Download對話框並開始下載該文件。

如何使用PHP提交表單並下載提供的文件？

這是我目前正在做的，以獲得必要的$ _POST信息。

//url for fhwa db file 
$fhwa_url = 'http://www.carriersoftwaredata.com/Login.aspx?file=FHWA'; 

//get the contents of the fhwa page 
$fhwa_login = file_get_contents($fhwa_url); 

//load contents of fhwa page into a DOMDocument object 
$fhwa_dom = new DOMDocument; 
if (!$fhwa_dom->loadhtml($fhwa_login)) 
{ 
    echo 'Could not Load html for FHWA Login page.'; 
} 
else 
{ 
    //create a post array to send back to the server - keys relate to the name of the input 
    $fhwa_post_items = array(
     '__VIEWSTATE'=>'', 
     'Email'=>'', 
     'Password'=>'', 
     '__EVENTVALIDATION'=>'', 
    ); 

    //create an xpath object 
    $xpath = new DOMXpath($fhwa_dom); 

    //iterate through the form1 form and find all inputs 
    foreach($xpath->query('//form[@name="form1"]//input') as $input) 
    { 
     //get name and value of input 
     $input_name = $input->getAttribute('name'); 
     $input_value = $input->getAttribute('value'); 

     //check if input name matches a key in the post array 
     if(array_key_exists($input_name, $fhwa_post_items)) 
     { 
     //if the input name is Email or Password enter the defined email and password 
     switch($input_name) 
     { 
      case 'Email': 
       $input_value = $email; 
       break; 
      case 'Password': 
       $input_value = $pass; 
       break; 
     }//switch 

     //assign value to post array 
     $fhwa_post[$input_name] = $input_value; 
     }// if 
    }// foreach 
}// if 

這就是我如何提交表單 - 但它似乎並沒有以我需要的方式工作。我希望stream_get_contents返回的內容是我想要下載的CSV文件的內容。

//get the url data and open a connection to the page 
    $url_data = parse_url($fhwa_url); 
    $post_str = http_build_query($fhwa_post); 

    //create socket 
    $fp = @fsockopen($url_data['host'], 80, $errno, $errstr, 30); 
    fputs($fp, "POST $fhwa_url HTTP/1.0\r\n"); 
    fputs($fp, "Host: {$url_data['host']}\r\n"); 
    fputs($fp, "User-Agent: Mozilla/4.5 [en]\r\n"); 
    fputs($fp, "Content-Type: application/x-www-form-urlencoded\r\n"); 
    fputs($fp, "Content-Length: ".strlen($post_str)."\r\n"); 
    fputs($fp, "\r\n"); 
    fputs($fp, $post_str."\r\n\r\n"); 
    echo stream_get_contents($fp); 
    fclose($fp); 

任何幫助是絕對讚賞。

Answer 1

我發現，我原來做了工作，但我根本沒有足夠發送的公告信息。我寫了一個函數來幫助我在將來輕鬆地做到這一點。

/** 
* @author: jeremysawesome - www.dnawebagency.com 
* @description: save_remote_file - Requires a url, a post array and a 
* filename. This function will send the post_array to the remote server 
* and retrieve the remote file. The retrieved remote file will be 
* saved to the filename specified by the filename parameter. 
* Note: This function should only be used when specific post information 
*  must be submitted to the remote server before the file download 
*  will begin. 
*  For Example: http://www.carriersoftwaredata.com/Login.aspx?file=FHWA 
* @param: post_url - the url to send a post request to 
* @param: post_array - the post arguments to send to the remote server. 
* @param: filename - the name to save the retrieved remote file as 
**/ 
function save_remote_file($post_url, $post_array, $filename) 
{ 
    //get the url data 
    $url_data = parse_url($post_url); 
    $post_str = http_build_query($post_array); 

    //build the headers to send to the form 
    $headers = "POST $post_url HTTP/1.0\r\n"; 
    $headers .= "Host: {$url_data['host']}\r\n"; 
    $headers .= "User-Agent: Mozilla/4.5 [en]\r\n"; 
    $headers .= "Content-Type: application/x-www-form-urlencoded\r\n"; 
    $headers .= "Content-Length: ".strlen($post_str)."\r\n"; 
    $headers .= "\r\n"; 
    $headers .= $post_str."\r\n\r\n"; 

    //create socket and download data 
    $fp = @fsockopen($url_data['host'], 80, $errno, $errstr, 30); 
    fputs($fp, $headers); 
    $remote_file = stream_get_contents($fp); 
    fclose($fp); 

    //save data 
    $saved_file = fopen($filename,'w') or die('Cannot Open File '. $filename); 
    fwrite($saved_file, $remote_file); 
    fclose($saved_file); 
}// save_remote_file 

對於那些關心 - 或需要做的，今後類似的東西，這裏是完整的代碼（沒有上述功能）。

---

<?php 
/** 
* @author: jeremysawesome - www.dnawebagency.com 
* @desciption: This file retrieves the database information from Carrier Software. 
* This should be run with a cron in order to download the files. The file works 
* by getting the contents of the Carrier Software login page for the database. An 
* array of post values is created based on the contents of the login page and the 
* defined username and password. A socket is opened and the post information is 
* passed to the login page. 
**/ 

//define username and pass 
$user_info = array(
    'user'=>'[USERNAME]', 
    'pass'=>'[PASSWORD]' 
); 

//url for fhwa db file 
$fhwa_url = 'http://www.carriersoftwaredata.com/Login.aspx?file=FHWA'; 

//get the contents of the fhwa page 
$fhwa_login = file_get_contents($fhwa_url); 

//load contents of fhwa page into a DOMDocument object 
$fhwa_dom = new DOMDocument; 
if (!$fhwa_dom->loadhtml($fhwa_login)) 
{ 
    die('Could not Load html for FHWA Login page.'); 
} 
else 
{ 
    //create a post array to send back to the server - keys relate to the name of the input - this allows us to retrieve the randomly generated values of hidden inputs 
    $fhwa_post_items = array(
     '__EVENTTARGET' => '', 
     '__EVENTARGUMENT' => '', 
     '__VIEWSTATE' => '', 
     'Email' => '', 
     'Password' => '', 
     'btnSubmit' => 'Submit', 
     '__EVENTVALIDATION' => '', 
    ); 

    //create an xpath object 
    $xpath = new DOMXpath($fhwa_dom); 

    //iterate through the form1 form and find all inputs 
    foreach($xpath->query('//form[@name="form1"]//input') as $input) 
    { 
     //get name and value of input 
     $input_name = $input->getAttribute('name'); 
     $input_value = $input->getAttribute('value'); 

     //check if input name matches a key in the post array 
     if(array_key_exists($input_name, $fhwa_post_items)) 
     { 
     //if the input name is Email or Password enter the defined email and password 
     switch($input_name) 
     { 
      case 'Email': 
       $input_value = $user_info['user']; 
       break; 
      case 'Password': 
       $input_value = $user_info['pass']; 
       break; 
     }//switch 

     //assign value to post array 
     $fhwa_post[$input_name] = $input_value; 
     }// if 
    }// foreach 

    //save the file - function shown above 
    save_remote_file($fhwa_url, $fhwa_post, "my_data_folder/my_fhwa-db.zip"); 
}// if 

Answer 2

看起來你需要使用捲曲庫：http://www.php.net/manual/en/book.curl.php

下面是他們使用後的數據與捲曲的網站爲例（在提交的表單數據是數組$數據）：

<?php 

/* http://localhost/upload.php: 
print_r($_POST); 
print_r($_FILES); 
*/ 

$ch = curl_init(); 

$data = array('name' => 'Foo', 'file' => '@/home/user/test.png'); 

curl_setopt($ch, CURLOPT_URL, 'http://localhost/upload.php'); 
curl_setopt($ch, CURLOPT_POST, 1); 
curl_setopt($ch, CURLOPT_POSTFIELDS, $data); 

curl_exec($ch); 
?> 

Answer 3

我將不得不這樣做的方法是使用捲曲。如果服務器正在加載，您可以將HTTP基本身份驗證與請求一起傳遞。之後，您應該能夠從curl請求中獲取數據並將其保存到本地機器，並使用php fwrite函數。

這是我用過的一個例子。我剔除了我的幻想邏輯，只是基本的。它也沒有用於基本身份驗證的代碼，但您應該能夠輕鬆地在Google中找到該代碼。我認爲這可能會比較容易，除非有什麼其他的事情你必須做。

// Download data 
$ch = curl_init('http://www.server.com/file.txt'); 
curl_setopt($ch, CURLOPT_HEADER, 0); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt($ch, CURLOPT_BINARYTRANSFER, 1); 
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); 
$rawdata = curl_exec($ch); 
curl_close ($ch); 

// Save data 
$fp = fopen('new/file/location.txt','w'); 
fwrite($fp, $rawdata); 
fclose($fp);