編寫csv到DSX Python 2.7筆記本的Ibm bluemix對​​象存儲器

Question

我正在嘗試從DSX Python筆記本將一個熊貓數據框編寫爲CSV到Bluemix Object Storage。我首先將數據框保存到「本地」CSV文件。然後我有一個例程，試圖將文件寫入對象存儲。我得到一個413響應 - 對象太大。該文件只有大約3MB。這裏是我的代碼的基礎上，JSON例如我發現這裏：http://datascience.ibm.com/blog/working-with-object-storage-in-data-science-experience-python-edition/

import requests 

def put_file(credentials, local_file_name): 
    """This function writes file content to Object Storage V3 """ 
    url1 = ''.join(['https://identity.open.softlayer.com', '/v3/auth/tokens']) 
    data = {'auth': {'identity': {'methods': ['password'], 
     'password': {'user': {'name': credentials['name'],'domain': {'id': credentials['domain']}, 
     'password': credentials['password']}}}}} 
    headers = {'Content-Type': 'text/csv'} 
    with open(local_file_name, 'rb') as f: 
     resp1 = requests.post(url=url1, data=f, headers=headers) 
    return resp1 

任何幫助或指針是非常讚賞。

Answer 1

這段代碼片段來自tutorial對我來說工作得很好（對於一個12 MB的文件）。

from io import BytesIO 
import requests 
import json 
import pandas as pd 

def put_file(credentials, local_file_name): 
    """This functions returns a StringIO object containing 
    the file content from Bluemix Object Storage V3.""" 
    f = open(local_file_name,'r') 
    my_data = f.read() 
    url1 = ''.join(['https://identity.open.softlayer.com', '/v3/auth/tokens']) 
    data = {'auth': {'identity': {'methods': ['password'], 
      'password': {'user': {'name': credentials['username'],'domain': {'id': credentials['domain_id']}, 
      'password': credentials['password']}}}}} 
    headers1 = {'Content-Type': 'application/csv'} 
    resp1 = requests.post(url=url1, data=json.dumps(data), headers=headers1) 
    resp1_body = resp1.json() 
    for e1 in resp1_body['token']['catalog']: 
     if(e1['type']=='object-store'): 
      for e2 in e1['endpoints']: 
         if(e2['interface']=='public'and e2['region']=='dallas'): 
          url2 = ''.join([e2['url'],'/', credentials['container'], '/', local_file_name]) 
    s_subject_token = resp1.headers['x-subject-token'] 
    headers2 = {'X-Auth-Token': s_subject_token, 'accept': 'application/json'} 
    resp2 = requests.put(url=url2, headers=headers2, data = my_data) 
    print resp2 

我創建使用隨機大熊貓數據幀：

df = pd.DataFrame(np.random.randint(0,100,size=(1000000, 4)), columns=list('ABCD')) 

其保存到csv

df.to_csv('myPandasData_1000000.csv',index=False) 

，然後把它放到對象存儲

put_file(credentials_1,'myPandasData_1000000.csv') 

你可以得到單擊對象庫中的任何對象的insert to code -> Insert credentials以訪問對象。