OpenCV的：基於高斯混合模型

Question

我想使用OpenCV的EM算法顏色extraction.I正在使用基於OpenCV的例子文檔中下面的代碼做顏色提取：

cv::Mat capturedFrame (height, width, CV_8UC3); 
int i, j; 
int nsamples = 1000; 
cv::Mat samples (nsamples, 2, CV_32FC1); 
cv::Mat labels; 
cv::Mat img = cv::Mat::zeros (height, height, CV_8UC3); 
img = capturedFrame; 
cv::Mat sample (1, 2, CV_32FC1); 
CvEM em_model; 
CvEMParams params; 
samples = samples.reshape (2, 0); 

    for (i = 0; i < N; i++) 
    {   
     //from the training samples 
     cv::Mat samples_part = samples.rowRange (i*nsamples/N, (i+1)*nsamples/N); 

     cv::Scalar mean (((i%N)+1)*img.rows/(N1+1),((i/N1)+1)*img.rows/(N1+1)); 
     cv::Scalar sigma (30,30); 
     cv::randn(samples_part,mean,sigma);      

    }  

    samples = samples.reshape (1, 0); 

    //initialize model parameters 
    params.covs   = NULL; 
    params.means  = NULL; 
    params.weights  = NULL; 
    params.probs  = NULL; 
    params.nclusters = N; 
    params.cov_mat_type = CvEM::COV_MAT_SPHERICAL; 
    params.start_step = CvEM::START_AUTO_STEP; 
    params.term_crit.max_iter = 300; 
    params.term_crit.epsilon = 0.1; 
    params.term_crit.type = CV_TERMCRIT_ITER|CV_TERMCRIT_EPS;  
    //cluster the data 
    em_model.train (samples, Mat(), params, &labels);  

    cv::Mat probs; 
    probs = em_model.getProbs(); 

    cv::Mat weights; 
    weights = em_model.getWeights(); 

cv::Mat modelIndex = cv::Mat::zeros (img.rows, img.cols, CV_8UC3); 

for (i = 0; i < img.rows; i ++) 
{ 
    for (j = 0; j < img.cols; j ++) 
    { 
     sample.at<float>(0) = (float)j; 
    sample.at<float>(1) = (float)i;  

    int response = cvRound (em_model.predict (sample)); 
    modelIndex.data [ modelIndex.cols*i + j] = response; 

    } 
} 

這裏我的問題是：

首先，我想提取每個模型，這裏總共五個，然後將這些相應的像素值存儲在五個不同的矩陣。在這種情況下，我可以分別使用五種不同的顏色。在這裏我只獲得了他們的索引，有沒有什麼方法可以在這裏實現他們相應的顏色？爲了簡單起見，我可以從找到基於這五種GMM的主色調開始。

其次，這裏我的示例數據點是「100」，它需要將近3秒。但我想在不超過30毫秒的時間內完成所有這些事情。我知道使用GMM的OpenCV背景提取非常快速，低於20ms，這意味着，我必須在30毫秒內爲所有600x800 = 480000像素完成所有這些操作。我發現predict函數是最耗時的。

Answer 1

第一個問題：

爲了做到顏色提取首先需要與你的輸入像素來訓練他們。之後，您只需再次遍歷所有輸入像素並使用predict（）對它們中的每一個進行分類。我已經附上了一個小例子，它利用EM來進行基於顏色的前景/背景分離。它向您展示瞭如何提取每個高斯的主色（平均值）以及如何訪問原始像素的顏色。

#include <opencv2/opencv.hpp> 

int main(int argc, char** argv) { 

    cv::Mat source = cv::imread("test.jpg"); 

    //ouput images 
    cv::Mat meanImg(source.rows, source.cols, CV_32FC3); 
    cv::Mat fgImg(source.rows, source.cols, CV_8UC3); 
    cv::Mat bgImg(source.rows, source.cols, CV_8UC3); 

    //convert the input image to float 
    cv::Mat floatSource; 
    source.convertTo(floatSource, CV_32F); 

    //now convert the float image to column vector 
    cv::Mat samples(source.rows * source.cols, 3, CV_32FC1); 
    int idx = 0; 
    for (int y = 0; y < source.rows; y++) { 
     cv::Vec3f* row = floatSource.ptr<cv::Vec3f > (y); 
     for (int x = 0; x < source.cols; x++) { 
      samples.at<cv::Vec3f > (idx++, 0) = row[x]; 
     } 
    } 

    //we need just 2 clusters 
    cv::EMParams params(2); 
    cv::ExpectationMaximization em(samples, cv::Mat(), params); 

    //the two dominating colors 
    cv::Mat means = em.getMeans(); 
    //the weights of the two dominant colors 
    cv::Mat weights = em.getWeights(); 

    //we define the foreground as the dominant color with the largest weight 
    const int fgId = weights.at<float>(0) > weights.at<float>(1) ? 0 : 1; 

    //now classify each of the source pixels 
    idx = 0; 
    for (int y = 0; y < source.rows; y++) { 
     for (int x = 0; x < source.cols; x++) { 

      //classify 
      const int result = cvRound(em.predict(samples.row(idx++), NULL)); 
      //get the according mean (dominant color) 
      const double* ps = means.ptr<double>(result, 0); 

      //set the according mean value to the mean image 
      float* pd = meanImg.ptr<float>(y, x); 
      //float images need to be in [0..1] range 
      pd[0] = ps[0]/255.0; 
      pd[1] = ps[1]/255.0; 
      pd[2] = ps[2]/255.0; 

      //set either foreground or background 
      if (result == fgId) { 
       fgImg.at<cv::Point3_<uchar> >(y, x, 0) = source.at<cv::Point3_<uchar> >(y, x, 0); 
      } else { 
       bgImg.at<cv::Point3_<uchar> >(y, x, 0) = source.at<cv::Point3_<uchar> >(y, x, 0); 
      } 
     } 
    } 

    cv::imshow("Means", meanImg); 
    cv::imshow("Foreground", fgImg); 
    cv::imshow("Background", bgImg); 
    cv::waitKey(0); 

    return 0; 
} 

我用以下圖像測試了代碼，它表現相當不錯。

第二個問題：

我注意到集羣的最大數量對性能產生巨大的影響。因此，最好將其設置爲非常保守的值，而不要將其設置爲空或將其設置爲像示例中那樣的樣本數。此外，該文件提到了一個迭代過程，用較少約束的參數重複優化模型。也許這給你一些加速。要閱讀更多內容，請查看爲train（）here提供的示例代碼內的文檔。