將日期時間格式轉換爲Unix時間戳Pandas

我試圖將正常的日期時間轉換爲熊貓中的unix時間戳。同時尋找一些樣本我只能找到一個例子here，但我不能在我的上下文中使用。該數據集沒有標題，最後的2 columns需要轉換UNIX time stamp並與前3列一起生成新的輸出。將日期時間格式轉換爲Unix時間戳Pandas

1466f7b93975983f6e292a8a4faaa4b2,1619b4d0d283c0dddb17d24a359a3b49,36db348cde68592a31d502366fc52932,2010-03-08 17:09:00.472544,2010-03-12 16:09:58.122987 
367c13356a5d22158f0ae56977134e2c,eedb7d0714796b64767a8710ea3844a7,925476200929fd346ea312cbe9a046fe,2010-03-08 17:08:29.174236,2010-03-12 16:09:58.122987 
edf6b1e4f67b0e8a5080d299c9f9aeb2,7cb7681b90388a7522d0f06578591567,ffde0649a72ded8e33522c503a4d5cbe,2010-03-08 17:08:22.030524,2010-03-12 16:09:58.122987 
6bb2ad8bc78897e99072d4d76cf0f19c,b644947ac4db03bdb518cfa71765f8c8,eb25089d396c06255cbb5f1bad801cc4,2010-03-08 17:07:55.819137,2010-03-12 16:09:58.122987

輸入文件擁有數百萬行，只有少數我已經發布在這裏。任何建議將是有價值的。在此先感謝

來源

2016-05-12 Sitz Blogz

您可以先read_csv然後將最後兩列轉換爲除以10**9。對於寫入文件時使用to_csv：

import pandas as pd 
import numpy as np 
import io 

temp=u"""1466f7b93975983f6e292a8a4faaa4b2,1619b4d0d283c0dddb17d24a359a3b49,36db348cde68592a31d502366fc52932,2010-03-08 17:09:00.472544,2010-03-12 16:09:58.122987 
367c13356a5d22158f0ae56977134e2c,eedb7d0714796b64767a8710ea3844a7,925476200929fd346ea312cbe9a046fe,2010-03-08 17:08:29.174236,2010-03-12 16:09:58.122987 
edf6b1e4f67b0e8a5080d299c9f9aeb2,7cb7681b90388a7522d0f06578591567,ffde0649a72ded8e33522c503a4d5cbe,2010-03-08 17:08:22.030524,2010-03-12 16:09:58.122987 
6bb2ad8bc78897e99072d4d76cf0f19c,b644947ac4db03bdb518cfa71765f8c8,eb25089d396c06255cbb5f1bad801cc4,2010-03-08 17:07:55.819137,2010-03-12 16:09:58.122987"""

#after testing replace io.StringIO(temp) to filename 
df = pd.read_csv(io.StringIO(temp), 
       header=None, #no header in csv 
       names=['a','b','c','d', 'e'], #set custom column names 
       parse_dates=['d','e']) #parse columns d, e to datetime 
print df 
            a         b \ 
0 1466f7b93975983f6e292a8a4faaa4b2 1619b4d0d283c0dddb17d24a359a3b49 
1 367c13356a5d22158f0ae56977134e2c eedb7d0714796b64767a8710ea3844a7 
2 edf6b1e4f67b0e8a5080d299c9f9aeb2 7cb7681b90388a7522d0f06578591567 
3 6bb2ad8bc78897e99072d4d76cf0f19c b644947ac4db03bdb518cfa71765f8c8 

            c       d \ 
0 36db348cde68592a31d502366fc52932 2010-03-08 17:09:00.472544 
1 925476200929fd346ea312cbe9a046fe 2010-03-08 17:08:29.174236 
2 ffde0649a72ded8e33522c503a4d5cbe 2010-03-08 17:08:22.030524 
3 eb25089d396c06255cbb5f1bad801cc4 2010-03-08 17:07:55.819137 

          e 
0 2010-03-12 16:09:58.122987 
1 2010-03-12 16:09:58.122987 
2 2010-03-12 16:09:58.122987 
3 2010-03-12 16:09:58.122987 


df['d'] = df['d'].astype(np.int64) // 10**9 
df['e'] = df['e'].astype(np.int64) // 10**9

print df 
            a         b \ 
0 1466f7b93975983f6e292a8a4faaa4b2 1619b4d0d283c0dddb17d24a359a3b49 
1 367c13356a5d22158f0ae56977134e2c eedb7d0714796b64767a8710ea3844a7 
2 edf6b1e4f67b0e8a5080d299c9f9aeb2 7cb7681b90388a7522d0f06578591567 
3 6bb2ad8bc78897e99072d4d76cf0f19c b644947ac4db03bdb518cfa71765f8c8 

            c   d   e 
0 36db348cde68592a31d502366fc52932 1268068140 1268410198 
1 925476200929fd346ea312cbe9a046fe 1268068109 1268410198 
2 ffde0649a72ded8e33522c503a4d5cbe 1268068102 1268410198 
3 eb25089d396c06255cbb5f1bad801cc4 1268068075 1268410198 

df.to_csv('filename', header=None, index=False)

來源

2016-05-12 16:22:21 jezrael

太感謝你了..讓我與文件讀取運行它，並回信。 –

再次感謝您的編輯。我是熊貓初學者..通過閱讀CSV文件的方式不是問題，但寫入輸出文件是;） –

沒問題，我編輯解決方案。 – jezrael

Unix的日期時間正好是自1月1日的秒數，從1970年正確

所以要保證轉換日期：

def dt2ut(dt): 
    epoch = pd.to_datetime('1970-01-01') 
    return (dt - epoch).total_seconds()

然後

import pandas as pd 
import numpy as np 
import io 

temp=u"""1466f7b93975983f6e292a8a4faaa4b2,1619b4d0d283c0dddb17d24a359a3b49,36db348cde68592a31d502366fc52932,2010-03-08 17:09:00.472544,2010-03-12 16:09:58.122987 
367c13356a5d22158f0ae56977134e2c,eedb7d0714796b64767a8710ea3844a7,925476200929fd346ea312cbe9a046fe,2010-03-08 17:08:29.174236,2010-03-12 16:09:58.122987 
edf6b1e4f67b0e8a5080d299c9f9aeb2,7cb7681b90388a7522d0f06578591567,ffde0649a72ded8e33522c503a4d5cbe,2010-03-08 17:08:22.030524,2010-03-12 16:09:58.122987 
6bb2ad8bc78897e99072d4d76cf0f19c,b644947ac4db03bdb518cfa71765f8c8,eb25089d396c06255cbb5f1bad801cc4,2010-03-08 17:07:55.819137,2010-03-12 16:09:58.122987""" 
#after testing replace io.StringIO(temp) to filename 
df = pd.read_csv(io.StringIO(temp), header=None, names=['a','b','c','d', 'e']) 

df['d'] = df['d'].apply(dt2ut).astype(np.int64) 
df['e'] = df['e'].apply(dt2ut).astype(np.int64)

來源

2016-05-12 16:27:39 piRSquared

我嘗試比較解決方案，但我們的輸出是不同的......你能檢查你的解決方案嗎？ – jezrael

我的appologies，轉換功能上的錯字。屬性'秒'應該是方法'total_seconds（）' – piRSquared

非常感謝你的解決方案。 :) –

將日期時間格式轉換爲Unix時間戳Pandas

回答

相關問題