caffe損失是nan或0

Question

我正在訓練一個網絡，我已經改爲從0.1到0.00001的學習率。輸出始終保持不變。沒有意思是用於訓練。 造成這種奇怪損失的原因是什麼？

I1107 15:07:28.381621 12333 solver.cpp:404]  Test net output #0: loss = 3.37134e+11 (* 1 = 3.37134e+11 loss) 
I1107 15:07:28.549142 12333 solver.cpp:228] Iteration 0, loss = 1.28092e+11 
I1107 15:07:28.549201 12333 solver.cpp:244]  Train net output #0: loss = 1.28092e+11 (* 1 = 1.28092e+11 loss) 
I1107 15:07:28.549211 12333 sgd_solver.cpp:106] Iteration 0, lr = 1e-07 
I1107 15:07:59.490077 12333 solver.cpp:228] Iteration 50, loss = -nan 
I1107 15:07:59.490170 12333 solver.cpp:244]  Train net output #0: loss = 0 (* 1 = 0 loss) 
I1107 15:07:59.490176 12333 sgd_solver.cpp:106] Iteration 50, lr = 1e-07 
I1107 15:08:29.177093 12333 solver.cpp:228] Iteration 100, loss = -nan 
I1107 15:08:29.177119 12333 solver.cpp:244]  Train net output #0: loss = 0 (* 1 = 0 loss) 
I1107 15:08:29.177125 12333 sgd_solver.cpp:106] Iteration 100, lr = 1e-07 
I1107 15:08:59.758381 12333 solver.cpp:228] Iteration 150, loss = -nan 
I1107 15:08:59.758513 12333 solver.cpp:244]  Train net output #0: loss = 0 (* 1 = 0 loss) 
I1107 15:08:59.758545 12333 sgd_solver.cpp:106] Iteration 150, lr = 1e-07 
I1107 15:09:30.210208 12333 solver.cpp:228] Iteration 200, loss = -nan 
I1107 15:09:30.210304 12333 solver.cpp:244]  Train net output #0: loss = 0 (* 1 = 0 loss) 
I1107 15:09:30.210310 12333 sgd_solver.cpp:106] Iteration 200, lr = 1e-07 

Answer 1

你的損失不是0，甚至沒有接近。你從​​（即〜10^11）開始，它看起來很快爆炸後，你得到nan。你需要大幅縮減你的損失值。如果您使用的是"EuclideanLoss"，則可能需要根據深度圖的大小對損失進行平均，將預測值縮放至[-1,1]範圍，或採用任何其他縮放方法來防止爆炸造成的損失。