如何在data.table中的組內連續創建時間序列？

Question

我有一個data.table包含來自不同位置（站點）的小時觀測的時間序列。在每個序列中都有缺口 - 缺少小時。我想填寫每個站點的小時序列，因此每個小時都有一行（儘管數據將丟失，不適用）。

實施例的數據：

library(data.table) 
library(lubridate) 

DT <- data.table(site = rep(LETTERS[1:2], each = 3), 
       date = ymd_h(c("2017080101", "2017080103", "2017080105", 
           "2017080103", "2017080105", "2017080107")), 
       # x = c(1.1, 1.2, 1.3, 2.1, 2.2, 2.3, 3.1, 3.2, 3.3), 
       x = c(1.1, 1.2, 1.3, 2.1, 2.2, 2.3), 
       key = c("site", "date")) 
DT 
# site    date x 
# 1: A 2017-08-01 01:00:00 1.1 
# 2: A 2017-08-01 03:00:00 1.2 
# 3: A 2017-08-01 05:00:00 1.3 
# 4: B 2017-08-01 03:00:00 2.1 
# 5: B 2017-08-01 05:00:00 2.2 
# 6: B 2017-08-01 07:00:00 2.3 

期望的結果DT2將包含所有的第一（最低）日期和每個站點中的最後（最大）日期之間的時間，其中x缺少其中新的行被插入：

# site    date x 
# 1: A 2017-08-01 01:00:00 1.1 
# 2: A 2017-08-01 02:00:00 NA 
# 3: A 2017-08-01 03:00:00 1.2 
# 4: A 2017-08-01 04:00:00 NA 
# 5: A 2017-08-01 05:00:00 1.3 
# 6: B 2017-08-01 03:00:00 2.1 
# 7: B 2017-08-01 04:00:00 NA 
# 8: B 2017-08-01 05:00:00 2.2 
# 9: B 2017-08-01 06:00:00 NA 
#10: B 2017-08-01 07:00:00 2.3 

我曾嘗試加入DT與min(date)和max(date)構建一個日期序列。這是在正確的方向，但日期範圍是所有的網站，而不是對每個單獨的站點，換行有缺失的部位，排序順序（密鑰）是錯誤的：

DT[.(seq(from = min(date), to = max(date), by = "hour")), 
    .SD, on="date"] 
# site    date x 
# 1: A 2017-08-01 01:00:00 1.1 
# 2: NA 2017-08-01 02:00:00 NA 
# 3: A 2017-08-01 03:00:00 1.2 
# 4: B 2017-08-01 03:00:00 2.1 
# 5: NA 2017-08-01 04:00:00 NA 
# 6: A 2017-08-01 05:00:00 1.3 
# 7: B 2017-08-01 05:00:00 2.2 
# 8: NA 2017-08-01 06:00:00 NA 
# 9: B 2017-08-01 07:00:00 2.3 

所以我很自然嘗試加入by = site：

DT[.(seq(from = min(date), to = max(date), by = "hour")), 
    .SD, on="date", by=.(site)] 
# site    date x 
# 1: A 2017-08-01 01:00:00 1.1 
# 2: A 2017-08-01 03:00:00 1.2 
# 3: A 2017-08-01 05:00:00 1.3 
# 4: NA    <NA> NA 
# 5: B 2017-08-01 03:00:00 2.1 
# 6: B 2017-08-01 05:00:00 2.2 
# 7: B 2017-08-01 07:00:00 2.3 

但是這也行不通。任何人都可以提出正確的data.table公式來給出所需的填充DT2如上所示？

Answer 1

library(data.table) 
library(lubridate) 
setDT(DT) 
test <- DT[, .(date = seq(min(date), max(date), by = 'hour')), by = 
       'site'] 
DT <- merge(test, DT, by = c('site', 'date'), all.x = TRUE) 


DT 
    site    date x 
1: A 2017-08-01 01:00:00 1.1 
2: A 2017-08-01 02:00:00 NA 
3: A 2017-08-01 03:00:00 1.2 
4: A 2017-08-01 04:00:00 NA 
5: A 2017-08-01 05:00:00 1.3 
6: B 2017-08-01 03:00:00 2.1 
7: B 2017-08-01 04:00:00 NA 
8: B 2017-08-01 05:00:00 2.2 
9: B 2017-08-01 06:00:00 NA 
10: B 2017-08-01 07:00:00 2.3 

Answer 2

感謝Frank和Wen讓我走上正軌。我找到了一個緊湊的data.table解決方案。結果DT2也是在網站和日期上鍵入的，就像在輸入表中一樣（儘管我沒有在OP中請求這個）。這是對Wen的解決方案的一個重新表述，採用data.table語法，我認爲這對於大型數據集來說會更有效率。

DT2 <- DT[setkey(DT[, .(date = seq(from = min(date), to = max(date), 
         by = "hour")), by = site], site, date), ] 
DT2 
# site    date x 
# 1: A 2017-08-01 01:00:00 1.1 
# 2: A 2017-08-01 02:00:00 NA 
# 3: A 2017-08-01 03:00:00 1.2 
# 4: A 2017-08-01 04:00:00 NA 
# 5: A 2017-08-01 05:00:00 1.3 
# 6: B 2017-08-01 03:00:00 2.1 
# 7: B 2017-08-01 04:00:00 NA 
# 8: B 2017-08-01 05:00:00 2.2 
# 9: B 2017-08-01 06:00:00 NA 
#10: B 2017-08-01 07:00:00 2.3 
key(DT2) 
# [1] "site" "date" 

EDIT1：如Frank所述，on=語法也可以使用。以下DT3公式提供了正確答案，但DT3未被鍵入，而DT2結果是鍵入的。這意味着如果需要鍵控結果，則需要「額外」setkey()。

DT3 <- DT[DT[, .(date = seq(from = min(date), to = max(date), 
        by = "hour")), by = site], on = c("site", "date"), ] 
DT3 
# site    date x 
# 1: A 2017-08-01 01:00:00 1.1 
# 2: A 2017-08-01 02:00:00 NA 
# 3: A 2017-08-01 03:00:00 1.2 
# 4: A 2017-08-01 04:00:00 NA 
# 5: A 2017-08-01 05:00:00 1.3 
# 6: B 2017-08-01 03:00:00 2.1 
# 7: B 2017-08-01 04:00:00 NA 
# 8: B 2017-08-01 05:00:00 2.2 
# 9: B 2017-08-01 06:00:00 NA 
#10: B 2017-08-01 07:00:00 2.3 
key(DT3) 
# NULL 
all.equal(DT2, DT3) 
# [1] "Datasets has different keys. 'target': site, date. 'current' has no key." 
all.equal(DT2, DT3, check.attributes = FALSE) 
# [1] TRUE 

是否有寫DT3表達給一個鍵結果的方式，比用明確其他setkey()？

編輯2：弗蘭克的評論暗示使用keyby = .EACHI的附加配方DT4。在這種情況下，.SD被插入爲j，這是在使用by或keyby時需要的。這給出了正確的答案，並且結果如同DT2公式那樣是關鍵的。

DT4 <- DT[DT[, .(date = seq(from = min(date), to = max(date), by = "hour")), 
      by = site], on = c("site", "date"), .SD, keyby = .EACHI] 
DT4 
# site    date x 
# 1: A 2017-08-01 01:00:00 1.1 
# 2: A 2017-08-01 02:00:00 NA 
# 3: A 2017-08-01 03:00:00 1.2 
# 4: A 2017-08-01 04:00:00 NA 
# 5: A 2017-08-01 05:00:00 1.3 
# 6: B 2017-08-01 03:00:00 2.1 
# 7: B 2017-08-01 04:00:00 NA 
# 8: B 2017-08-01 05:00:00 2.2 
# 9: B 2017-08-01 06:00:00 NA 
#10: B 2017-08-01 07:00:00 2.3 
key(DT4) 
# [1] "site" "date" 
identical(DT2, DT4) 
# [1] TRUE