基於eval_metric停止xgboost

Question

我想運行xgboost來解決非常嘈雜的功能問題，並且希望根據我定義的自定義eval_metric來停止輪次數。

基於領域知識我知道，當eval_metric（在訓練數據上評估）高於某個值時xgboost過度擬合。我想只是在特定的輪次上採用擬合模型，而不是繼續進行。

什麼是最好的方法來實現這一目標？

這將有點符合早期停止標準，但不完全符合。

或者，如果有可能從中間輪獲得模型？

下面是一個更好地解釋問題的例子。 （使用附帶xgboost幫助文檔的玩具爲例，選擇默認eval_metric）

library(xgboost) 
data(agaricus.train, package='xgboost') 
train <- agaricus.train 
bstSparse <- xgboost(data = train$data, label = train$label, max.depth = 2, eta = 1, nthread = 2, nround = 5, objective = "binary:logistic") 

這裏是輸出

[0] train-error:0.046522 
[1] train-error:0.022263 
[2] train-error:0.007063 
[3] train-error:0.015200 
[4] train-error:0.007063 

現在讓我們從領域知識說，我知道，一旦列車誤差低於0.015（在這種情況下是第三輪），任何進一步的回合只會導致過度擬合。我將如何在第三輪後停止訓練過程並獲得訓練好的模型，以便將它用於不同數據集的預測？

我需要在許多不同的數據集上運行訓練過程，並且我沒有意識到可能需要多少輪才能獲得低於固定數量的錯誤，因此我無法將nrounds參數設置爲預定義值。我的唯一直覺是，一旦訓練錯誤低於一個數字，我需要停止進一步的訓練。

Answer 1

# In the absence of any code you have tried or any data you are using then try something like this: 

require(xgboost) 
library(Metrics) # for rmse to calculate errors 

# Assume you have a training set db.train and have some feature indices of interest and a test set db.test 
    predz <- c(2,4,6,8,10,12) 
    predictors <- names(db.train[,predz]) 
# you have some response you are interested in 
    outcomeName <- "myLabel" 

# you may like to include for testing some other parameters like: eta, gamma, colsample_bytree, min_child_weight 

    # here we look at depths from 1 to 4 and rounds 1 to 100 but set your own values 

     smallestError <- 100 # set to some sensible value depending on your eval metric 

     for (depth in seq(1,4,1)) { 
     for (rounds in seq(1,100,1)) { 

        # train 
        bst <- xgboost(data = as.matrix(db.train[,predictors]), 
            label = db.train[,outcomeName], 
            max.depth = depth, nround = rounds, 
            eval_metric = "logloss", 
            objective = "binary:logistic", verbose=TRUE) 
        gc() 

        # predict 
        predictions <- as.numeric(predict(bst, as.matrix(db.test[,predictors]), outputmargin=TRUE)) 
        err <- rmse(as.numeric(db.test[,outcomeName]), as.numeric(predictions)) 

        if (err < smallestError) { 
          smallestError = err 
          print(paste(depth,rounds,err)) 
        }  
       } 
     } 

# You could adapt this code for your particular evaluation metric and print this out to suit your situation. Similarly you could introduce a break in the code when some specified number of rounds is reached that satisfies some condition you seek to achieve.