我想用Encog庫來訓練一個神經網絡。Encog:重量不斷增加
數據集(〜7000個示例)分裂(爲訓練(60%),交叉驗證(20%)和測試(20%))之前是線性歸一化,使得其看起來像這樣:
Min=-1.000000; Max=1.000000; Average=-0.077008
目標(理想)的數據集(也線性歸一化)看起來是這樣的:
Min=0.201540; Max=0.791528; Average=0.477080
我initialie網絡是這樣的:
mNetwork = new BasicNetwork();
mNetwork.addLayer(new BasicLayer(null, false, trainingDataSet.getInputSize()));
mNetwork.addLayer(new BasicLayer(new ActivationSigmoid(), true, numberOfNeurons));
mNetwork.addLayer(new BasicLayer(new ActivationSigmoid(), false, trainingDataSet.getIdealSize()));
mNetwork.getStructure().finalizeStructure();
mNetwork.reset();
我用ResilientPropagation
教練(試過Backpropagation
以及):
ResilientPropagation training = new ResilientPropagation(mNetwork, mTrainingDataSet);
for (int i = 0; i < mNumberOfIterations; ++i) {
training.iteration();
result.trainingErrors[i] = mNetwork.calculateError(mTrainingDataSet);
result.validationErrors[i] = mNetwork.calculateError(mValidationDataSet);
System.out.println(String.format("Iteration #%d: error=%.8f", i, training.getError()));
}
training.finishTraining();
在由教練培訓報告錯誤的過程一般下降。在完成訓練後,我傾倒重量:
0.04274211002929323,-0.5481902707068103,0.28978635361541294,-0.203635994176051,22965.18656660482,22964.992410871928,22966.23882308963,22966.355722230965,22965.036733143017,22964.894030965166,22966.002332259202,22965.177650526788,22966.009842504238,22965.971560546248,22966.257180159628,22966.234150681423,-21348.311232865744,-21640.843082085466,-21057.13217475862,-21347.52051343582,-21347.988714647887,-21641.161098510198,-21057.27275747668,-21348.784123049118,-21347.719149090022,-21639.773689115867,-21057.095487328377,-21348.269878600076,22800.304816865206,23090.894751729396,22799.39388588725,22799.72408290791,22800.249806096508,22799.19823789763,22799.85510732227,22799.99965531053,22799.574773588192,22799.57945236908,22799.12542315293,22799.523065957797
它們通常不是很大就是很小。乙狀結腸它最終與預測收斂到一定數目,例如權重(經過500次迭代得到)以上給我:
Min=0.532179; Max=0.532179; Average=0.532179
似乎有一些錯誤與網絡或培訓的配置。如果我的網絡的差異很小,至少它會產生一個與目標範圍相符的結果。如果它有很大的差異,它會匹配目標。而現在,它完全錯過了目標。
即使預測結束,錯誤如何減少並變得相當低?任何人在我上面的例子中看到明顯的錯誤?我對神經網絡還不是很有經驗。
的權重會增加,我看到訓練的神經網絡,大的權重。我沒有看到上面的代碼有任何問題。你正在向正常範圍正常化。至於爲什麼培訓不收斂,這往往是複雜的回答。常見的嫌疑人包括:偏斜的數據(太多類似的結果),嘈雜的數據,矛盾的結果。 – JeffHeaton 2014-09-14 00:25:12
正如我所提到的,我對神經網絡沒有經驗。我認爲噪音只會降低準確性。我相信噪音可能是問題。我會盡量減少它。 – Pijusn 2014-09-14 05:26:17