通過使用sigmoid函數組合線性迴歸的方式進行邏輯迴歸

Question

我想實現Logistic迴歸算法而不調用任何matlab支持的函數和後綴我稱之爲logistic迴歸的matlab函數mnrfit因此我可以交叉確認我的算法運行良好。

我正在執行的過程如下。我首先創建一個具有輸入數據的向量x和一個向量y [0,1]，該向量對於每個數據x都具有相應的類。我使用梯度下降法對這些數據進行線性迴歸，一旦我提取係數，我將通過S形函數傳遞該線。稍後，我對x = 10進行預測，以找出此輸入的第1類的可能性。簡單，因爲..

之後，我打電話給matlab函數mnrfit並提取邏輯迴歸的係數。爲了做出同樣的預測，我將函數mnrval的參數設爲10，因爲我想像以前一樣預測輸入x = 10。我的結果是不同的，我不知道爲什麼..

這兩個繪圖顯示了每種情況下的概率密度函數顯示在最後。

我也附上了實施的代碼。

% x is the continues input and y is the category of every output [1 or 0] 
x = (1:100)'; % independent variables x(s) 
y(1:10) = 0; % Dependent variables y(s) -- class 0 
y(11:100) = 1; % Dependent variables y(s) -- class 1 
y=y'; 
y = y(randperm(length(y))); % Random order of y array 
x=[ones(length(x),1) x]; % This is done for vectorized code 

%% Initialize Linear regression parameters 

m = length(y); % number of training examples 
% initialize fitting parameters - all zeros 
Alpha = 0; % gradient 
Beta = 0; % offset 
% Some gradient descent settings 
% iterations must be a big number because we are taking very small steps . 
iterations = 100000; 
% Learning step must be small because the line must fit the data between 
% [0 and 1] 
Learning_step_a = 0.0005; % step parameter 

%% Run Gradient descent 

fprintf('Running Gradient Descent ...\n') 
for iter = 1:iterations 
% In every iteration calculate objective function 
h= Alpha.*x(:,2)+ Beta.*x(:,1); 
% Update line variables 
Alpha=Alpha - Learning_step_a * (1/m)* sum((h-y).* x(:,2)); 
Beta=Beta - Learning_step_a * (1/m) * sum((h-y).*x(:,1)); 
end 

% This is my linear Model 
LinearModel=Alpha.*x(:,2)+ Beta.*x(:,1); 
% I pass it through a sigmoid ! 
LogisticRegressionPDF = 1 ./ (1 + exp(-LinearModel)); 
% Make a prediction for p(y==1|x==10) 
Prediction1=LogisticRegressionPDF(10); 

%% Confirmation with matlab function mnrfit 

B=mnrfit(x(:,2),y+1); % Find Logistic Regression Coefficients 
mnrvalPDF = mnrval(B,x(:,2)); 
% Make a prediction .. p(y==1|x==10) 
Prediction2=mnrvalPDF(10,2); 

%% Plotting Results 

% Plot Logistic Regression Results ... 
figure; 
plot(x(:,2),y,'g*'); 
hold on 
plot(x(:,2),LogisticRegressionPDF,'k--'); 
hold off 
title('My Logistic Regression PDF') 
xlabel('continues input'); 
ylabel('propability density function'); 

% Plot Logistic Regression Results (mnrfit) ...  
figure,plot(x(:,2),y,'g*'); 
hold on 
plot(x(:,2),mnrvalPDF(:,2),'--k') 
hold off 
title('mnrval Logistic Regression PDF') 
xlabel('continues input'); 
ylabel('propability density function') 

爲什麼我的圖（只要預測）針對每種情況是不一樣的？

• 您可能會提取的輸出在每次執行時會有所不同，因爲y向量中的1和0的順序是隨機的。

Answer 1

我使用梯度下降法發展出自己的迴歸算法。對於「良好」的訓練數據，我的算法別無選擇，只能與mnrfit一樣收斂。對於「不太好」的訓練數據，我的算法沒有用mnrfit關閉。係數和相關模型可以很好地預測結果，但不如mnrfit。繪製殘差顯示mnrfit的殘差接近於零（0.00001），而9x10-200與接近零的殘差幾乎爲零。我嘗試改變阿爾法，步數和初始theta猜測，但這樣做只會產生不同的theta結果。當我用一個好的數據集調整這些參數時，我的theta開始與mnrfit更好地融合。

Answer 2

非常感謝用戶信息user3779062。 PDF文件裏面是我想要的。我已經實現了隨機梯度下降，所以爲了實現Logistic迴歸，唯一的區別是通過for循環中的sigmoid函數更新假設函數，並且只要更新thetas規則中的符號更改順序。結果與mnrval相同。我實現了很多示例的代碼，結果大部分時間都是相同的（特別是如果數據集很好，並且在這兩個類中都有很多信息）。我附上最終的代碼和結果集的隨機結果。

% Machine Learning : Logistic Regression 

% Logistic regression is working as linear regression but as an output 
% specifies the propability to be attached to one category or the other. 
% At the beginning we created a well defined data set that can be easily 
% be fitted by a sigmoid function. 

clear all; close all; clc; 

% This example runs many times to compare a lot of results 
for examples=1:10:100 
clearvars -except examples 

%% Creatte Training Data 

% x is the continues input and y is the category of every output [1 or 0] 
x = (1:100)'; % independent variables x(s) 
y(1:examples) = 0; % Dependent variables y(s) -- class 0 
y(examples+1:100) = 1; % Dependent variables y(s) -- class 1 
y=y'; 
y = y(randperm(length(y))); % Random order of y array 
x=[ones(length(x),1) x]; % This is done for vectorized code 

%% Initialize Linear regression parameters 

m = length(y); % number of training examples 
% initialize fitting parameters - all zeros 
Alpha = 0; % gradient 
Beta = 0; % offset 
% Some gradient descent settings 
% iterations must be a big number because we are taking very small steps . 
iterations = 100000; 
% Learning step must be small because the line must fit the data between 
% [0 and 1] 
Learning_step_a = 0.0005; % step parameter 

%% Run Gradient descent 

fprintf('Running Gradient Descent ...\n') 
for iter = 1:iterations 

% Linear hypothesis function 
h= Alpha.*x(:,2)+ Beta.*x(:,1); 

% Non - Linear hypothesis function 
hx = 1 ./ (1 + exp(-h)); 

% Update coefficients 
Alpha=Alpha + Learning_step_a * (1/m)* sum((y-hx).* x(:,2)); 
Beta=Beta + Learning_step_a * (1/m) * sum((y-hx).*x(:,1)); 

end 

% Make a prediction for p(y==1|x==10) 
Prediction1=hx(10) 

%% Confirmation with matlab function mnrfit 

B=mnrfit(x(:,2),y+1); % Find Logistic Regression Coefficients 
mnrvalPDF = mnrval(B,x(:,2)); 
% Make a prediction .. p(y==1|x==10) 
Prediction2=mnrvalPDF(10,2) 

%% Plotting Results 

% Plot Logistic Regression Results ... 
figure; 
subplot(1,2,1),plot(x(:,2),y,'g*'); 
hold on 
subplot(1,2,1),plot(x(:,2),hx,'k--'); 
hold off 
title('My Logistic Regression PDF') 
xlabel('continues input'); 
ylabel('propability density function'); 

% Plot Logistic Regression Results (mnrfit) ...  
subplot(1,2,2),plot(x(:,2),y,'g*'); 
hold on 
subplot(1,2,2),plot(x(:,2),mnrvalPDF(:,2),'--k') 
hold off 
title('mnrval Logistic Regression PDF') 
xlabel('continues input'); 
ylabel('propability density function')  
end 

結果..

非常感謝！