2013-03-15 48 views
2

我的工作在數據幀的轉換,並在一前一後段錯誤

Previous Post

阿倫工作阿倫和裏卡多,提出了一個輝煌的解決方案(矩陣乘法)來實現我試圖做。

這解決方案爲小數據集像我在例子中提到,現在我運行它具有以下尺寸的數據幀在同一個解決方案:

Total rows: 143345 
Total Persons: 98461 
Total Items : 30 

現在,當我運行以下命令

A <- acast(Person~Item+BorS,data=df,fun.aggregate=length,drop=FALSE) 

我得到這個錯誤..

Error: segfault from C stack overflow 

這是becau se,我沒有足夠的處理/記憶能力。我的機器有4 GB RAM,2.8 GHz i7處理器(Macbook)?我們如何處理這些類型的案件?

+0

多少「BorS」的級別有(是B還是S?) – mnel 2013-03-15 02:17:09

+0

您是否嘗試過使用稀疏矩陣(Matrix包)? – ndoogan 2013-03-15 02:38:14

+0

是的只有兩個級別B和S.不,我沒有嘗試過稀疏矩陣,現在我會閱讀它。 – user2171177 2013-03-15 03:18:49

回答

4

A data.table解決方案。這是通過聚合,然後再創建新的data.table,並通過引用填充

library(data.table) 

# some sample data 
DT <- data.table(Person = sample(98461, 144000, replace = TRUE), item = sample(c(letters,LETTERS[1:4]), 144000, replace = TRUE), BorS = sample(c('B','S'), 144000, replace = TRUE)) 
# aggregate to get the number of rows in each subgroup by list item and BorS 
# the `length` of each subgroup 
DTl <- DT[,.N , by = list(Person, item, BorS)] 
# the columns you want to create 
newn <- sort(DT[, do.call(paste0,do.call(expand.grid,list(unique(item),unique(BorS))))]) 
# create a column which has this id combination in DTl 
DTl[, comnb := paste0(item, BorS)] 
# set the key so we can join/subset easily 
setkey(DTl, comnb) 
# create a data.table that has 1 row for each person, and has columns for all the combinations 
# of item and BorS 
DTb <- DTl[, list(Person)][, c(newn) := 0L] 
# set the key so we can join/subset easily 
setkey(DTb, Person) 
# this bit could be far quicker, but I think 
# would require a feature request to data.table 
for(nn in newn){ 
    # for each of the cominations extract which persons have 
    # this combination >0 
    pp <- DTl[list(nn), list(Person,N)] 
    # for the people who have N > 0 
    # assign the correct numbers in the correct column in DTb 
    DTb[list(pp[['Person']]), c(nn) := pp[['N']]] 
} 

要完成你initital問題,您可以提取DTb相應列的矩陣

A <- DTb[,-1,with = FALSE] 

results <- crossprod(A) 
+0

你能否從第二行開始解釋你的解決方案? – user2171177 2013-03-15 03:27:56

+0

@ user2171177 - 我希望這樣做更有意義。 – mnel 2013-03-15 03:32:38

+0

對不起,我的無知... – user2171177 2013-03-15 03:53:41