2012-12-28 13 views
9

關注此維基百科文章SQL join我想清楚地瞭解我們如何加入data.table。 在這個過程中,我們可能在加入NAs時發現了一個錯誤。 以維基例如:data.table內部/外部連接與類型雙重錯誤的連接列中的NA?

R) X = data.table(name=c("Raf","Jon","Ste","Rob","Smi","Joh"),depID=c(31,33,33,34,34,NA),key="depID") 
R) Y = data.table(depID=c(31,33,34,35),depName=c("Sal","Eng","Cle","Mar"),key="depID") 
R) X 
    name depID 
1: Joh NA 
2: Raf 31 
3: Jon 33 
4: Ste 33 
5: Rob 34 
6: Smi 34 
R) Y 
    depID depName 
1: 31  Sal 
2: 33  Eng 
3: 34  Cle 
4: 35  Mar 

LEFT OUTER JOIN

R) merge.data.frame(X,Y,all.x=TRUE) 
    depID name depName 
1 31 Raf  Sal 
2 33 Jon  Eng 
3 33 Ste  Eng 
4 34 Rob  Cle 
5 34 Smi  Cle 
6 NA Joh <NA> 

merge.data.table不輸出相同的結果,並顯示什麼,我認爲是在河旁2

錯誤
R) merge(X,Y,all.x=TRUE) 
    depID name depName 
1: NA Joh  Eng 
2: 31 Raf  NA 
3: 33 Jon  Eng 
4: 33 Ste  Eng 
5: 34 Rob  Cle 
6: 34 Smi  Cle 
R) Y[X] #same -> :(
    depID depName name 
1: NA  Eng Joh 
2: 31  NA Raf 
3: 33  Eng Jon 
4: 33  Eng Ste 
5: 34  Cle Rob 
6: 34  Cle Smi 

RIGHT OUTER JOIN 貌似相同

R) merge.data.frame(X,Y,all.y=TRUE) 
    depID name depName 
1 31 Raf  Sal 
2 33 Jon  Eng 
3 33 Ste  Eng 
4 34 Rob  Cle 
5 34 Smi  Cle 
6 35 <NA>  Mar 

R) merge(X,Y,all.y=TRUE) 
    depID name depName 
1: NA Joh  Eng 
2: 31 NA  Sal 
3: 33 Jon  Eng 
4: 33 Ste  Eng 
5: 34 Rob  Cle 
6: 34 Smi  Cle 
7: 35 NA  Mar 

INNER(自然科學)JOIN

R) merge.data.frame(X,Y) 
    depID name depName 
1 31 Raf  Sal 
2 33 Jon  Eng 
3 33 Ste  Eng 
4 34 Rob  Cle 
5 34 Smi  Cle 
R) merge(X,Y) 
    depID name depName 
1: NA Joh  Eng 
2: 33 Jon  Eng 
3: 33 Ste  Eng 
4: 34 Rob  Cle 
5: 34 Smi  Cle 

回答

7

是的,它看起來像有關的(尷尬)新的bug NA在關鍵。有關NA的其他討論關鍵是不可能的,但我沒有意識到它可能會以這種方式搞砸。將調查。謝謝...

#2453 NA in double key column messes up joins (NA in integer and character ok)

現在固定在1.8.7(提交780),從消息:double類型的連接列

NA可能導致兩個X [Y]和合並(X,Y)返回不正確的結果,#2453。由於C源代碼中錯誤的x == NA_REAL,應該是ISNA(x)。對keyed連接的雙重支持是對data.table的一個相對最新的補充,但是同樣令人尷尬。修正並添加了測試。非常感謝statquant的徹底和可重複的報告。

+0

正如報道中以前,刪除答案(這實際上是一個評論),如果depID列是整數,則合併工作正常。 –

+0

@MatthewLundberg有趣,謝謝你。爲什麼它被刪除,聽起來很有用!這可以解釋爲什麼測試沒有抓住它 - 我可能只想到用整數來測試NA,認爲NA是雙倍不會有所作爲。 –

+0

我無法回答,但不知道你是否會嘗試與沒有提示的整數合併。回答者聲稱角色也有效,但我沒有核實。 –

2

在其他答案的評論跟進,是的,這裏是證明它不僅影響鍵入double列(NA在integercharacter列都OK)。

X = data.table(name=c("Raf","Jon","Ste","Rob","Smi","Joh"), 
       depID=as.integer(c(31,33,33,34,34,NA)),key="depID") 
Y = data.table(depID=as.integer(c(31,33,34,35)), 
       depName=c("Sal","Eng","Cle","Mar"),key="depID") 
Y[X] 
    depID depName name 
1: NA  NA Joh 
2: 31  Sal Raf 
3: 33  Eng Jon 
4: 33  Eng Ste 
5: 34  Cle Rob 
6: 34  Cle Smi 

merge.data.frame(X,Y,all.x=T) 
    depID name depName 
1 31 Raf  Sal 
2 33 Jon  Eng 
3 33 Ste  Eng 
4 34 Rob  Cle 
5 34 Smi  Cle 
6 NA Joh <NA> 

Y = data.table(depID=as.character(c(31,33,34,35)), 
       depName=c("Sal","Eng","Cle","Mar"),key="depID") 
X = data.table(name=c("Raf","Jon","Ste","Rob","Smi","Joh"), 
       depID=as.character(c(31,33,33,34,34,NA)),key="depID") 
X 
    name depID 
1: Raf 31 
2: Jon 33 
3: Ste 33 
4: Rob 34 
5: Smi 34 
6: Joh NA 
Y 
    depID depName 
1: 31  Sal 
2: 33  Eng 
3: 34  Cle 
4: 35  Mar 
str(X) 
Classes ‘data.table’ and 'data.frame': 6 obs. of 2 variables: 
$ name : chr "Raf" "Jon" "Ste" "Rob" ... 
$ depID: chr "31" "33" "33" "34" ... 
- attr(*, "sorted")= chr "depID" 
- attr(*, ".internal.selfref")=<externalptr> 

merge.data.frame(X,Y,all.x=T) 
    depID name depName 
1 31 Raf  Sal 
2 33 Jon  Eng 
3 33 Ste  Eng 
4 34 Rob  Cle 
5 34 Smi  Cle 
6 <NA> Joh <NA> 

Y[X] 
    depID depName name 
1: 31  Sal Raf 
2: 33  Eng Jon 
3: 33  Eng Ste 
4: 34  Cle Rob 
5: 34  Cle Smi 
6: NA  NA Joh 

的問題已得到修復馬修DOWLE IN V.1.8.7

1

一些信息,可以是有用的:

library(data.table); 

X <- data.table(name=c("Raf","Jon","Ste","Rob","Smi","Joh"),depID=c(31,33,33,34,34,NA),key="depID") 
#R) X 
    #name depID 
#1: Joh NA 
#2: Raf 31 
#3: Jon 33 
#4: Ste 33 
#5: Rob 34 
#6: Smi 34 

Y <- data.table(depID=c(31,33,34,35),depName=c("Sal","Eng","Cle","Mar"),key="depID") 
#R) Y 
    #depID depName 
#1: 31  Sal 
#2: 33  Eng 
#3: 34  Cle 
#4: 35  Mar 

################# 
#LEFT OUTER JOIN# 
################# 
LJ <- merge.data.frame(X,Y,by="depID",all.x=TRUE); #by is implicit (see ?merge.data.frame) 
#R) LJ 
    #depID name depName 
#1 31 Raf  Sal 
#2 33 Jon  Eng 
#3 33 Ste  Eng 
#4 34 Rob  Cle 
#5 34 Smi  Cle 
#6 NA Joh <NA> 

LJ2 <- Y[X]; 
#R) LJ2 
    #depID depName name 
#1: NA  NA Joh 
#2: 31  Sal Raf 
#3: 33  Eng Jon 
#4: 33  Eng Ste 
#5: 34  Cle Rob 
#6: 34  Cle Smi 

################## 
#RIGHT OUTER JOIN# 
################## 
RJ <- merge.data.frame(X,Y,by="depID",all.y=TRUE); #by is implicit (see ?merge.data.frame) 
#R) RJ 
    #depID name depName 
#1 31 Raf  Sal 
#2 33 Jon  Eng 
#3 33 Ste  Eng 
#4 34 Rob  Cle 
#5 34 Smi  Cle 
#6 35 <NA>  Mar 

RJ2 <- X[Y]; 
#R) RJ2 
    #depID name depName 
#1: 31 Raf  Sal 
#2: 33 Jon  Eng 
#3: 33 Ste  Eng 
#4: 34 Rob  Cle 
#5: 34 Smi  Cle 
#6: 35 NA  Mar 

################# 
#FULL OUTER JOIN# 
################# 
FJ <- merge.data.frame(X,Y,all=T) 
#R) FJ 
    #depID name depName 
#1 31 Raf  Sal 
#2 33 Jon  Eng 
#3 33 Ste  Eng 
#4 34 Rob  Cle 
#5 34 Smi  Cle 
#6 35 <NA>  Mar 
#7 NA Joh <NA> 

FJ2 <- merge(X,Y,all=T) 
#R) FJ2 
    #depID name depName 
#1: NA Joh  NA 
#2: 31 Raf  Sal 
#3: 33 Jon  Eng 
#4: 33 Ste  Eng 
#5: 34 Rob  Cle 
#6: 34 Smi  Cle 
#7: 35 NA  Mar 

#################### 
#NATURAL INNER JOIN# 
#################### 
IJ <- merge.data.frame(X,Y) 
#R) IJ 
    #depID name depName 
#1 31 Raf  Sal 
#2 33 Jon  Eng 
#3 33 Ste  Eng 
#4 34 Rob  Cle 
#5 34 Smi  Cle 

IJ2 <- merge(X,Y) 
#R) IJ2 
    #depID name depName 
#1: 31 Raf  Sal 
#2: 33 Jon  Eng 
#3: 33 Ste  Eng 
#4: 34 Rob  Cle 
#5: 34 Smi  Cle 


A <- data.table(time=as.POSIXct(c("10:01:01","10:01:02","10:01:04","10:01:05","10:01:02","10:01:01","10:01:01"),format="%H:%M:%S"), 
       b=c("a","a","a","a","b","c","c"), 
       d=c(1,1.9,2,1.8,5,4.1,4.2)); 
B <- data.table(time=as.POSIXct(c("10:01:01","10:01:03","10:01:00","10:01:01"),format="%H:%M:%S"),b=c("a","a","c","d"), e=c(1L,2L,3L,4L)); 
setkey(A,b,time) 
setkey(B,b,time) 


########### 
#ASOF JOIN# 
########### 
AOJ <- B[A,roll=T] 
#R) AOJ 
    #b    time e d 
#1: a 2013-01-11 10:01:01 1 1.0 
#2: a 2013-01-11 10:01:02 1 1.9 
#3: a 2013-01-11 10:01:04 2 2.0 
#4: a 2013-01-11 10:01:05 2 1.8 
#5: b 2013-01-11 10:01:02 NA 5.0 
#6: c 2013-01-11 10:01:01 3 4.1 
#7: c 2013-01-11 10:01:01 3 4.2