scikit-learn SGDClassifier熱啓動忽略

Question

我試圖從scikit-learn版本0.15.1使用SGDClassifier。除了迭代次數之外，似乎沒有任何方法設置收斂標準。所以我想通過在每次迭代時檢查錯誤手動完成，然後熱啓動額外的迭代，直到改進足夠小。

不幸的是，warm_start標誌和coef_init/intercept_init似乎都沒有實際啓動優化 - 它們都似乎從頭開始。

我該怎麼辦？如果沒有真正的收斂標準或熱烈的開始，分類器不可用。

請注意以下在每次重新啓動時偏差如何增加很多，以及損失如何增加但隨着進一步的迭代而下降。經過250次迭代後，偏差爲-3.44，平均損失爲1.46。

sgd = SGDClassifier(loss='log', alpha=alpha, verbose=1, shuffle=True, 
        warm_start=True) 
print('INITIAL FIT') 
sgd.fit(X, y, sample_weight=sample_weight) 
sgd.n_iter = 1 
print('\nONE MORE ITERATION') 
sgd.fit(X, y, sample_weight=sample_weight) 
sgd.n_iter = 3 
print('\nTHREE MORE ITERATIONS') 
sgd.fit(X, y, sample_weight=sample_weight) 


INITIAL FIT 
-- Epoch 1 
Norm: 254.11, NNZs: 92299, Bias: -5.239955, T: 122956, Avg. loss: 28.103236 
Total training time: 0.04 seconds. 
-- Epoch 2 
Norm: 138.81, NNZs: 92598, Bias: -5.180938, T: 245912, Avg. loss: 16.420537 
Total training time: 0.08 seconds. 
-- Epoch 3 
Norm: 100.61, NNZs: 92598, Bias: -5.082776, T: 368868, Avg. loss: 12.240537 
Total training time: 0.12 seconds. 
-- Epoch 4 
Norm: 74.18, NNZs: 92598, Bias: -5.076395, T: 491824, Avg. loss: 9.859404 
Total training time: 0.17 seconds. 
-- Epoch 5 
Norm: 55.57, NNZs: 92598, Bias: -5.072369, T: 614780, Avg. loss: 8.280854 
Total training time: 0.21 seconds. 

ONE MORE ITERATION 
-- Epoch 1 
Norm: 243.07, NNZs: 92598, Bias: -11.271497, T: 122956, Avg. loss: 26.148746 
Total training time: 0.04 seconds. 

THREE MORE ITERATIONS 
-- Epoch 1 
Norm: 258.70, NNZs: 92598, Bias: -16.058395, T: 122956, Avg. loss: 29.666688 
Total training time: 0.04 seconds. 
-- Epoch 2 
Norm: 142.24, NNZs: 92598, Bias: -15.809559, T: 245912, Avg. loss: 17.435114 
Total training time: 0.08 seconds. 
-- Epoch 3 
Norm: 102.71, NNZs: 92598, Bias: -15.715853, T: 368868, Avg. loss: 12.731181 
Total training time: 0.12 seconds. 

Answer 1

warm_start=True將使用擬合係數爲出發點，但它重新開始學習稅率表。

如果你想手動檢查收斂我建議你使用partial_fit，而不是作爲fit建議@AdrienNK：

sgd = SGDClassifier(loss='log', alpha=alpha, verbose=1, shuffle=True, 
       warm_start=True, n_iter=1) 
sgd.partial_fit(X, y) 
# after 1st iteration 
sgd.partial_fit(X, y) 
# after 2nd iteration 
...