libsvm：使用leave-one-out評估SVM

Question

我正在嘗試使用libsvm和MATLAB來評估一個vs-all SVM，唯一的問題是我的數據集不夠大，無法選擇特定的測試集。因此，我想用leave-one-out來評估我的分類器。

我對使用支持向量機並不是特別有經驗，所以請原諒我，如果我有點困惑，應該怎麼做。我需要爲分類器生成精度與召回曲線以及混淆矩陣，但我不知道從哪裏開始。

我已經給出了它，並提出以下作爲一個粗略的開始做假離開訓練，但我不知道如何做評估。

function model = do_leave_one_out(labels, data) 
      acc = []; 
      bestC = []; 
      bestG = []; 
      for ii = 1:length(data) 
        % Training data for this iteration 
        trainData = data; 
        trainData(ii) = []; 
        looLabel = labels(ii); 
        trainLabels = labels; 
        trainLabels(ii) = []; 

        % Do grid search to find the best parameters? 

        acc(ii) = bestReportedAccuracy; 
        bestC(ii) = bestValueForC; 
        bestG(ii) = bestValueForG; 
      end 
      % After this I am not sure how to train and evaluate the final model 
end 

Answer 1

我想提供一些你可能感興趣的模塊，你可以將它們合併到你的函數中。希望能幫助到你。

留一出：

scrambledList = randperm(totalNumberOfData); 
trainingData = Data(scrambledList(1:end-1),:); 
trainingLabel = Label(scrambledList(1:end-1)); 
testData = Data(scrambledList(end),:); 
testLabel = Label(scrambledList(end)); 

網格搜索（雙級情況下）：

acc = 0; 
for log2c = -1:3, 
    for log2g = -4:1, 
    cmd = ['-v 5 -c ', num2str(2^log2c), ' -g ', num2str(2^log2g)]; 
    cv = svmtrain(trainingLabel, trainingData, cmd); 
    if (cv >= acc), 
     acc = cv; bestC = 2^log2c; bestG = 2^log2g; 
    end  
    end 
end 

一VS-所有（用於多類情況）：

model = cell(NumofClass,1); 
for k = 1:NumofClass 
    model{k} = svmtrain(double(trainingLabel==k), trainingData, '-c 1 -g 0.2 -b 1'); 
end 

%% calculate the probability of different labels 

pr = zeros(1,NumofClass); 
for k = 1:NumofClass 
    [~,~,p] = svmpredict(double(testLabel==k), testData, model{k}, '-b 1'); 
    pr(:,k) = p(:,model{k}.Label==1); %# probability of class==k 
end 

%% your label prediction will be the one with highest probability: 

[~,predctedLabel] = max(pr,[],2);