Python：通過URL的.csv循環並保存爲另一列

Question

Python新手，閱讀了一堆，並觀看了很多視頻。我無法讓它工作，我感到沮喪。

我有一個鏈接列表如下圖所示：

"KGS ID","Latitude","Longitude","Location","Operator","Lease","API","Elevation","Elev_Ref","Depth_start","Depth_stop","URL" 
"1002880800","37.2354869","-100.4607509","T32S R29W, Sec. 27, SW SW NE","Stanolind Oil and Gas Co.","William L. Rickers 1","15-119-00164","2705"," KB","2790","7652","http://www.kgs.ku.edu/WellLogs/32S29W/1043696830.zip" 
"1002880821","37.1234622","-100.1158111","T34S R26W, Sec. 2, NW NW NE","SKELLY OIL CO","GRACE MCKINNEY 'A' 1","15-119-00181","2290"," KB","4000","5900","http://www.kgs.ku.edu/WellLogs/34S26W/1043696831.zip" 

我試圖讓蟒蛇去「URL」，並將其保存在名爲「定位」爲文件名的文件夾中「API.las」 。

EX）...... 「位置」/組/ 「API」 .las C：//.../T32S R29W/Sec.27/15-119-00164.las

該文件有數百行和鏈接供下載。我想實現一個休眠功能也不會轟炸服務器。

有什麼不同的方式來做到這一點？我試過熊貓和其他一些方法......有什麼想法？

Answer 1

你將不得不做這樣的

for link, file_name in zip(links, file_names): 
    u = urllib.urlopen(link) 
    udata = u.read() 
    f = open(file_name+".las", w) 
    f.write(udata) 
    f.close() 
    u.close() 

的東西，如果你的文件的內容是你想要什麼沒有，你可能想看看在刮圖書館像BeautifulSoup解析。

Answer 2

方法1： -

您的文件已經假設1000行。通過這個masterlist>
[ROW1，ROW2，ROW3等]

一旦這樣做，環 -

創建masterlist其具有存儲在該形式的數據。每次迭代你都會得到一個字符串格式的行。 拆分它製作一個列表並拼接最後一列url，即行[-1]

並將其追加到名爲result_url的空列表中。一旦它運行的所有行，它保存在一個文件，你可以方便地使用os模塊創建一個目錄，然後將您的文件那邊

方法2： -

如果文件過於龐大，閱讀在try塊中一行一行地處理你的數據（使用csv模塊，你可以將每一行作爲一個列表，拼接url並將它每次寫入文件API.las）。

一旦你的程序移動了第1001行，它將移動到除了可以「通過」或寫入打印以獲得通知的塊之外。
在方法2中，您不是將所有數據保存在任何數據結構中，只是在執行時存儲單行，因此速度更快。

import csv, os 
    directory_creater = os.mkdir('Locations')  
    fme = open('./Locations/API.las','w+') 
    with open('data.csv','r') as csvfile: 
     spamreader = csv.reader(csvfile, delimiter = ',') 
     print spamreader.next() 
     while True: 
      try: 
       row= spamreader.next() 
       get_url = row[-1] 
       to_write = get_url+'\n' 
       fme.write(to_write) 
      except: 
       print "Program has run. Check output." 
       exit(1) 

此代碼可以在更短的時間內完成所有您提到的效率。

Answer 3

這可能有點骯髒，但這是解決問題的第一步。這一切都取決於CSV中的每個值都包含在雙引號中。如果這不是真的，這個解決方案將需要大量調整。

代碼：

import os 

csv = """ 
"KGS ID","Latitude","Longitude","Location","Operator","Lease","API","Elevation","Elev_Ref","Depth_start","Depth_stop","URL" 
"1002880800","37.2354869","-100.4607509","T32S R29W, Sec. 27, SW SW NE","Stanolind Oil and Gas Co.","William L. Rickers 1","15-119-00164","2705"," KB","2790","7652","http://www.kgs.ku.edu/WellLogs/32S29W/1043696830.zip" 
"1002880821","37.1234622","-100.1158111","T34S R26W, Sec. 2, NW NW NE","SKELLY OIL CO","GRACE MCKINNEY 'A' 1","15-119-00181","2290"," KB","4000","5900","http://www.kgs.ku.edu/WellLogs/34S26W/1043696831.zip" 
""".strip() # trim excess space at top and bottom 

root_dir = '/tmp/so_test' 

lines = csv.split('\n') # break CSV on newlines 
header = lines[0].strip('"').split('","') # grab first line and consider it the header 

lines_d = [] # we're about to perform the core actions, and we're going to store it in this variable 
for l in lines[1:]: # we want all lines except the top line, which is a header 
    line_broken = l.strip('"').split('","') # strip off leading and trailing double-quote 
    line_assoc = zip(header, line_broken) # creates a tuple of tuples out of the line with the header at matching position as key 
    line_dict = dict(line_assoc) # turn this into a dict 
    lines_d.append(line_dict) 

    section_parts = [s.strip() for s in line_dict['Location'].split(',')] # break Section value to get pieces we need 

    file_out = os.path.join(root_dir, '%s%s%s%sAPI.las'%(section_parts[0], os.path.sep, section_parts[1], os.path.sep)) # format output filename the way I think is requested 

    # stuff to show what's actually put in the files 
    print file_out, ':' 
    print ' ', '"%s"'%('","'.join(header),) 
    print ' ', '"%s"'%('","'.join(line_dict[h] for h in header)) 

輸出：

~/so_test $ python so_test.py 
/tmp/so_test/T32S R29W/Sec. 27/API.las : 
    "KGS ID","Latitude","Longitude","Location","Operator","Lease","API","Elevation","Elev_Ref","Depth_start","Depth_stop","URL" 
    "1002880800","37.2354869","-100.4607509","T32S R29W, Sec. 27, SW SW NE","Stanolind Oil and Gas Co.","William L. Rickers 1","15-119-00164","2705"," KB","2790","7652","http://www.kgs.ku.edu/WellLogs/32S29W/1043696830.zip" 
/tmp/so_test/T34S R26W/Sec. 2/API.las : 
    "KGS ID","Latitude","Longitude","Location","Operator","Lease","API","Elevation","Elev_Ref","Depth_start","Depth_stop","URL" 
    "1002880821","37.1234622","-100.1158111","T34S R26W, Sec. 2, NW NW NE","SKELLY OIL CO","GRACE MCKINNEY 'A' 1","15-119-00181","2290"," KB","4000","5900","http://www.kgs.ku.edu/WellLogs/34S26W/1043696831.zip" 
~/so_test $