0
我有一個混合專家代碼,其中使用少量的參數。這是因爲如下:專家混合 - fminunc優化過早停止
global x_au;
global x_vi;
global x_alpha;
global y;
global parameter;
options = optimoptions(@fminunc,'GradObj', 'on', 'Algorithm','quasi-newton','MaxIter', 10000,'Display','iter-detailed'); % change number of iterations
optTheta=[];
x_au=x_au_train;
x_vi=x_vi_train;
x_alpha=x_alpha_train;
y=y_train;
parameter=zeros(8969,1);
%expectation step
fprintf('opt1 begins');
opt_1;
fprintf('opt1 complete');
%maximaization step
[x] = fminunc(@costfunction,parameter(1:4483),options);
parameter(1:4483)=x;
resnorm1=total_error(parameter(1:4483));
k=1;
count = 1;
while(1)
opt_1;
fprintf('expectation complete');
%maximaization step
[x] = fminunc(@costfunction,parameter(1:4483),options);
parameter(1:4483)=x;
resnorm2=total_error(parameter(1:4483));
fprintf('resnorm1-resnorm2 - %f, resnorm2 - %f, k - %f',resnorm1-resnorm2,0.000001*resnorm2,k);
if((resnorm1-resnorm2)< .000001*resnorm2 & k~=1) %% to decrease training time
break;
end
但現在,當我必須有大量的參數上的問題用這個,我得到以下日誌。
First-order
Iteration Func-count f(x) Step-size optimality
0 1 5.31444e+10 4.75e+14
Optimization stopped because the objective function cannot be decreased in the
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.
First-order
Iteration Func-count f(x) Step-size optimality
0 1 5.31444e+10 4.75e+14
Optimization stopped because the objective function cannot be decreased in the
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.
resnorm1-resnorm2 - 0.000000, resnorm2 - 53144.356560, k - 1.000000
First-order
Iteration Func-count f(x) Step-size optimality
0 1 5.31444e+10 4.75e+14
Optimization stopped because the objective function cannot be decreased in the
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.
resnorm1-resnorm2 - 0.000000, resnorm2 - 53144.356560, k - 2.000000
>>
然後過程完成,結果很糟糕。至於,可以看出fminunc無法正確優化。有人能幫我一下嗎?
嗨,感謝您的幫助!我試圖正常化,沒有幫助。一階最優性遞減至2.99e + 10,但該過程在3次迭代後仍然停止。減少訓練速度係數是什麼意思?請解釋。 – Deven
此外,對於〜24000大小的數據集,模型使用〜9000參數是否合理? – Deven
我的意思是這些算法都接近相同:'x [n + 1] = x [n] + alpha * dF/dx'。看起來你的'alpha * dF/dx'太大了。我建議減少阿爾法。但是我在matlab優化包中不太好,alpha現在可以自動管理。 9000參數可能沒問題,那麼你需要引入正則化。 –