2013-07-25 33 views
0

多年來一直在這裏讀書,喜歡它。我自己的小問題的時間。Wget Cookie會話設置爲DUMMY,302重定向問題?

我們曾使用wget SSH從供應商的後端檢索產品信息(價格,可用性)。不幸的是,他們改變了他們的方式,癱瘓了我的數據流:/

我已經修復了一些,但我無法獲取所有數據,因爲我的cookie不能很好地存儲了。閱讀所有Google結果,嘗試所有結果,但我的知識欠缺。我會堅持餅乾設置的一部分,我敢打賭,如果這是固定的,我可以再次檢索所有文件的順序也很好。

這是我使用,我沒有信譽點,所以我不能添加多個鏈接,我會用「URL-這裏」作爲替補我https://www.supplier.com

該Cookie給出了這樣的:

# HTTP cookie file. 
# Generated by Wget on 2013-07-25 06:03:44. 
# Edit at your own risk. 

www.supplier.com FALSE / FALSE 0 PHPSESSID DUMMY 

通過瀏覽器一個cookie顯示會話的值,這部分似乎沒有再被寫爲:/

也試過這個,都具有相同的RESU LT:

/usr/local/bin/wget -O /dev/null --cookies=on --keep-session-cookies --save-cookies=cookiepp.txt --referer='url-here/auth/login.php' --post-data='login=123654&password=123654' 'url-here/auth/login.php' 

而且也試過這種直接與手動保存cookie來retireve:

wget --no-check-certificate --span-hosts --header "Cookie:PHPSESSID=h771tqr0spe8ufbq6fash2msf6" "url-here/sortment/s/?group="$i"&excel=1" 

最後,SSH命令行流量:

--2013-07-25 06:03:43-- https://www.supplier.com/auth/login.php 
Resolving www.supplier.com (www.supplier.com)... 00.000.000.0 
Connecting to www.supplier.com (www.supplier.com)|00.000.000.0|:443... connected. 
HTTP request sent, awaiting response... 302 Found 
Location: https://www.supplier.com/auth/login.php [following] 
--2013-07-25 06:03:44-- https://www.supplier.com/auth/login.php 
Reusing existing connection to www.supplier.com:443. 
HTTP request sent, awaiting response... 200 OK 
Length: 6884 (6.7K) [text/html] 
Saving to: '/dev/null' 

最後的調試信息:

Setting --output-document (outputdocument) to /dev/null 
Setting --cookies (cookies) to on 
Setting --keep-session-cookies (keepsessioncookies) to 1 
Setting --save-cookies (savecookies) to cookiepp.txt 
Setting --referer (referer) to https://www.supplier.com/auth/login.php 
Setting --post-data (postdata) to login=123654&password=123654 
DEBUG output created by Wget 1.14 on freebsd9.1. 

URI encoding = 'US-ASCII' 
--2013-07-25 07:01:30-- https://www.supplier.com/auth/login.php 
Resolving www.supplier.com (www.supplier.com)... 89.105.214.133 
Caching www.supplier.com => 89.105.214.133 
Connecting to www.supplier.com (www.supplier.com)|89.105.214.133|:443... connected. 
Created socket 5. 
Releasing 0x00000008021baa40 (new refcount 1). 
Initiating SSL handshake. 
Handshake successful; connected socket 5 to SSL handle 0x000000080204c400 
certificate: 
    subject: /C=NL/postalCode=7271LB/ST=Gelderland/L=Borculo/street=Hesselinks Es 11/O=Varuvo bv/OU= /OU=Hosted by WideXS/OU=InstantSSL/CN=www.supplier.com 
    issuer: /C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO High-Assurance Secure Server CA 
X509 certificate successfully verified and matches host www.supplier.com 

---request begin--- 
POST /auth/login.php HTTP/1.1 
Referer: https://www.supplier.com/auth/login.php 
User-Agent: Wget/1.14 (freebsd9.1) 
Accept: */* 
Host: www.supplier.com 
Connection: Keep-Alive 
Content-Type: application/x-www-form-urlencoded 
Content-Length: 32 

---request end--- 
[POST data: login=123654&password=123654] 
HTTP request sent, awaiting response... 
---response begin--- 
HTTP/1.1 302 Found 
Date: Thu, 25 Jul 2013 14:01:32 GMT 
Server: Apache 
P3P: CP="NOI DSP COR NID CURa OUR NOR STA" 
Set-Cookie: PHPSESSID=DUMMY; path=/ 
Status: 302 Found 
Location: https://www.supplier.com/auth/login.php 
Vary: Accept-Encoding 
Content-Length: 0 
Keep-Alive: timeout=15, max=100 
Connection: Keep-Alive 
Content-Type: text/html; charset=Windows-1252 

---response end--- 
302 Found 

Stored cookie www.supplier.com -1 (ANY)/<session> <insecure> [expiry none] PHPSESSID DUMMY 
Registered socket 5 for persistent reuse. 
URI content encoding = 'Windows-1252' 
Location: https://www.supplier.com/auth/login.php [following] 
] done. 
URI content encoding = None 
--2013-07-25 07:01:31-- https://www.supplier.com/auth/login.php 
Reusing existing connection to www.supplier.com:443. 
Reusing fd 5. 

---request begin--- 
GET /auth/login.php HTTP/1.1 
Referer: https://www.supplier.com/auth/login.php 
User-Agent: Wget/1.14 (freebsd9.1) 
Accept: */* 
Host: www.supplier.com 
Connection: Keep-Alive 
Cookie: PHPSESSID=DUMMY 

---request end--- 
HTTP request sent, awaiting response... 
---response begin--- 
HTTP/1.1 200 OK 
Date: Thu, 25 Jul 2013 14:01:32 GMT 
Server: Apache 
P3P: CP="NOI DSP COR NID CURa OUR NOR STA" 
Set-Cookie: PHPSESSID=DUMMY; path=/ 
Vary: Accept-Encoding 
Content-Length: 6884 
Keep-Alive: timeout=15, max=99 
Connection: Keep-Alive 
Content-Type: text/html; charset=Windows-1252 

---response end--- 
200 OK 
Deleted old cookie (to be replaced.) 

Stored cookie www.supplier.com -1 (ANY)/<session> <insecure> [expiry none] PHPSESSID DUMMY 
URI content encoding = 'Windows-1252' 
Length: 6884 (6.7K) [text/html] 
Saving to: '/dev/null' 

100%[===================================================================================================================>] 6,884  43.1KB/s in 0.2s 

2013-07-25 07:01:32 (43.1 KB/s) - '/dev/null' saved [6884/6884] 

Saving cookies to cookiepp.txt. 
Done saving cookies. 

我真的是我失去了這一點,在這一點上我希望得到別人的幫助。

回答

1

固定它使用捲曲:雖然

/usr/local/bin/curl -k -c cookie.txt -d "login_alias=464" -d "password=6446" -d "from=%2F" https://www.supplier.com/auth/login.php 
paste theCodesOnePerLine theNamesOnePerLine | 
while read i j 
do 
/usr/local/bin/curl -k -L -b cookie.txt -o /usr/home/$j.xls 'https://www.supplier.com/?group='$i'&excel=1' 
done 

感謝,對頁面空間:P