KDB + like asof加入熊貓時間序列數據？

Question

kdb +具有通常用於沿時間列連接表的aj函數。

這裏是一個例子，我有貿易和報價表，我得到每筆交易的主流報價。

q)5# t 
time   sym price size 
----------------------------- 
09:30:00.439 NVDA 13.42 60511 
09:30:00.439 NVDA 13.42 60511 
09:30:02.332 NVDA 13.42 100 
09:30:02.332 NVDA 13.42 100 
09:30:02.333 NVDA 13.41 100 

q)5# q 
time   sym bid ask bsize asize 
----------------------------------------- 
09:30:00.026 NVDA 13.34 13.44 3  16 
09:30:00.043 NVDA 13.34 13.44 3  17 
09:30:00.121 NVDA 13.36 13.65 1  10 
09:30:00.386 NVDA 13.36 13.52 21 1  
09:30:00.440 NVDA 13.4 13.44 15 17 

q)5# aj[`time; t; q] 
time   sym price size bid ask bsize asize 
----------------------------------------------------- 
09:30:00.439 NVDA 13.42 60511 13.36 13.52 21 1  
09:30:00.439 NVDA 13.42 60511 13.36 13.52 21 1  
09:30:02.332 NVDA 13.42 100 13.34 13.61 1  1  
09:30:02.332 NVDA 13.42 100 13.34 13.61 1  1  
09:30:02.333 NVDA 13.41 100 13.34 13.51 1  1 

如何使用熊貓做同樣的操作？我正在使用貿易和報價數據幀的索引是datetime64。

In [55]: quotes.head() 
Out[55]: 
           bid ask bsize asize 
2012-09-06 09:30:00.026000 13.34 13.44  3  16 
2012-09-06 09:30:00.043000 13.34 13.44  3  17 
2012-09-06 09:30:00.121000 13.36 13.65  1  10 
2012-09-06 09:30:00.386000 13.36 13.52  21  1 
2012-09-06 09:30:00.440000 13.40 13.44  15  17 

In [56]: trades.head() 
Out[56]: 
          price size 
2012-09-06 09:30:00.439000 13.42 60511 
2012-09-06 09:30:00.439000 13.42 60511 
2012-09-06 09:30:02.332000 13.42 100 
2012-09-06 09:30:02.332000 13.42 100 
2012-09-06 09:30:02.333000 13.41 100 

我看到熊貓有ASOF功能，但是未對數據幀定義，僅在系列對象。我想可以循環播放每一個系列，並逐一對齊，但我想知道是否有更好的方法？

Answer 1

正如你在問題中提到，通過每列循環應該爲你工作：

df1.apply(lambda x: x.asof(df2.index)) 

我們有可能創造DataFrame.asof更快南天真的版本做一次性的所有列。但現在，我認爲這是最直接的方法。

Answer 2

我前段時間寫了下廣告ordered_merge功能：

In [27]: quotes 
Out[27]: 
         time bid ask bsize asize 
0 2012-09-06 09:30:00.026000 13.34 13.44  3  16 
1 2012-09-06 09:30:00.043000 13.34 13.44  3  17 
2 2012-09-06 09:30:00.121000 13.36 13.65  1  10 
3 2012-09-06 09:30:00.386000 13.36 13.52  21  1 
4 2012-09-06 09:30:00.440000 13.40 13.44  15  17 

In [28]: trades 
Out[28]: 
         time price size 
0 2012-09-06 09:30:00.439000 13.42 60511 
1 2012-09-06 09:30:00.439000 13.42 60511 
2 2012-09-06 09:30:02.332000 13.42 100 
3 2012-09-06 09:30:02.332000 13.42 100 
4 2012-09-06 09:30:02.333000 13.41 100 

In [29]: ordered_merge(quotes, trades) 
Out[29]: 
         time bid ask bsize asize price size 
0 2012-09-06 09:30:00.026000 13.34 13.44  3  16 NaN NaN 
1 2012-09-06 09:30:00.043000 13.34 13.44  3  17 NaN NaN 
2 2012-09-06 09:30:00.121000 13.36 13.65  1  10 NaN NaN 
3 2012-09-06 09:30:00.386000 13.36 13.52  21  1 NaN NaN 
4 2012-09-06 09:30:00.439000 NaN NaN NaN NaN 13.42 60511 
5 2012-09-06 09:30:00.439000 NaN NaN NaN NaN 13.42 60511 
6 2012-09-06 09:30:00.440000 13.40 13.44  15  17 NaN NaN 
7 2012-09-06 09:30:02.332000 NaN NaN NaN NaN 13.42 100 
8 2012-09-06 09:30:02.332000 NaN NaN NaN NaN 13.42 100 
9 2012-09-06 09:30:02.333000 NaN NaN NaN NaN 13.41 100 

In [32]: ordered_merge(quotes, trades, fill_method='ffill') 
Out[32]: 
         time bid ask bsize asize price size 
0 2012-09-06 09:30:00.026000 13.34 13.44  3  16 NaN NaN 
1 2012-09-06 09:30:00.043000 13.34 13.44  3  17 NaN NaN 
2 2012-09-06 09:30:00.121000 13.36 13.65  1  10 NaN NaN 
3 2012-09-06 09:30:00.386000 13.36 13.52  21  1 NaN NaN 
4 2012-09-06 09:30:00.439000 13.36 13.52  21  1 13.42 60511 
5 2012-09-06 09:30:00.439000 13.36 13.52  21  1 13.42 60511 
6 2012-09-06 09:30:00.440000 13.40 13.44  15  17 13.42 60511 
7 2012-09-06 09:30:02.332000 13.40 13.44  15  17 13.42 100 
8 2012-09-06 09:30:02.332000 13.40 13.44  15  17 13.42 100 
9 2012-09-06 09:30:02.333000 13.40 13.44  15  17 13.41 100 

它可以很容易地（當然，有人誰是熟悉代碼）擴展爲「左連接」模仿KDB。我認識到在這種情況下，提前填寫交易數據是不合適的;只是說明功能。

Answer 3

pandas 0.19 has introduced an asof join：

pd.merge_asof(trades, quotes, on='time') 

的語義是非常類似於在Q/KDB +功能。