當OpenCV進程運行時QMainWindow沒有響應

Question

我在我的QtGui應用程序中使用自定義的OpenCV VideoProcessor-Class。我的MainWindow有2個ViewerWidgets，用於顯示由VideoProcessor對象產生的Input和Output幀。 VideoProcessor-Object在這些ViewerWidgets上採用指針來顯示這些Widget上的處理幀。

當我啓動應用程序時，GUI窗口中的所有內容都會響應用戶輸入。但是當我開始處理它停止響應。我甚至無法關閉窗口或從應用程序菜單中選擇一些東西。處理顯示正確的輸出並保持運行，但窗口不再響應。

這是主窗口的插槽，開始處理：

void MainWindow::on_actionStart_Capture_triggered() 
{ 
    // Create instance 
    p = new VideoProcessor(); 
      // Open video file 
      p->setInput(0); 
      // Declare a window to display the video 
      p->displayInput("Current Frame"); 
      p->displayOutput("Output Frame"); 
      // Play the video at the original frame rate 
      p->setDelay(1000./p->getFrameRate()); 
      // Set the frame processor callback function 
      p->setFrameProcessor(canny); 
      // Start the process 
      p->run(cvWidgetIn, cvWidgetOut); 
} 

這是，視頻處理器。該文件來自OpenCV Cookbook，我將其更改爲在下面的代碼末尾的run（）函數中指向我的ViewerWidgets。

#if !defined VPROCESSOR 
#define VPROCESSOR 

#include <iostream> 
#include <iomanip> 
#include <sstream> 
#include <string> 
#include <vector> 
#include <opencv2/core/core.hpp> 
#include <opencv2/highgui/highgui.hpp> 
#include "cvwidget.h" 

// The frame processor interface 
class FrameProcessor { 

    public: 
    // processing method 
    virtual void process(cv:: Mat &input, cv:: Mat &output)= 0; 
}; 

class VideoProcessor { 

    private: 

     // the OpenCV video capture object 
     cv::VideoCapture capture; 
     // the callback function to be called 
     // for the processing of each frame 
     void (*process)(cv::Mat&, cv::Mat&); 
     // the pointer to the class implementing 
     // the FrameProcessor interface 
     FrameProcessor *frameProcessor; 
     // a bool to determine if the 
     // process callback will be called 
     bool callIt; 
     // Input display window name 
     std::string windowNameInput; 
     // Output display window name 
     std::string windowNameOutput; 
     // delay between each frame processing 
     int delay; 
     // number of processed frames 
     long fnumber; 
     // stop at this frame number 
     long frameToStop; 
     // to stop the processing 
     bool stop; 

     // vector of image filename to be used as input 
     std::vector<std::string> images; 
     // image vector iterator 
     std::vector<std::string>::const_iterator itImg; 

     // the OpenCV video writer object 
     cv::VideoWriter writer; 
     // output filename 
     std::string outputFile; 

     // current index for output images 
     int currentIndex; 
     // number of digits in output image filename 
     int digits; 
     // extension of output images 
     std::string extension; 

     // to get the next frame 
     // could be: video file; camera; vector of images 
     bool readNextFrame(cv::Mat& frame) { 

      if (images.size()==0) 
       return capture.read(frame); 
      else { 

       if (itImg != images.end()) { 

        frame= cv::imread(*itImg); 
        itImg++; 
        return frame.data != 0; 
       } 
      } 
     } 

     // to write the output frame 
     // could be: video file or images 
     void writeNextFrame(cv::Mat& frame) { 

      if (extension.length()) { // then we write images 

       std::stringstream ss; 
       ss << outputFile << std::setfill('0') << std::setw(digits) << currentIndex++ << extension; 
       cv::imwrite(ss.str(),frame); 

      } else { // then write video file 

       writer.write(frame); 
      } 
     } 

    public: 

     // Constructor setting the default values 
     VideoProcessor() : callIt(false), delay(-1), 
      fnumber(0), stop(false), digits(0), frameToStop(-1), 
      process(0), frameProcessor(0) {} 

     // set the name of the video file 
     bool setInput(std::string filename) { 

     fnumber= 0; 
     // In case a resource was already 
     // associated with the VideoCapture instance 
     capture.release(); 
     images.clear(); 

     // Open the video file 
     return capture.open(filename); 
     } 

     // set the camera ID 
     bool setInput(int id) { 

     fnumber= 0; 
     // In case a resource was already 
     // associated with the VideoCapture instance 
     capture.release(); 
     images.clear(); 

     // Open the video file 
     return capture.open(id); 
     } 

     // set the vector of input images 
     bool setInput(const std::vector<std::string>& imgs) { 

     fnumber= 0; 
     // In case a resource was already 
     // associated with the VideoCapture instance 
     capture.release(); 

     // the input will be this vector of images 
     images= imgs; 
     itImg= images.begin(); 

     return true; 
     } 

     // set the output video file 
     // by default the same parameters than input video will be used 
     bool setOutput(const std::string &filename, int codec=0, double framerate=0.0, bool isColor=true) { 

      outputFile= filename; 
      extension.clear(); 

      if (framerate==0.0) 
       framerate= getFrameRate(); // same as input 

      char c[4]; 
      // use same codec as input 
      if (codec==0) { 
       codec= getCodec(c); 
      } 

      // Open output video 
      return writer.open(outputFile, // filename 
       codec, // codec to be used 
       framerate,  // frame rate of the video 
       getFrameSize(), // frame size 
       isColor);  // color video? 
     } 

     // set the output as a series of image files 
     // extension must be ".jpg", ".bmp" ... 
     bool setOutput(const std::string &filename, // filename prefix 
      const std::string &ext, // image file extension 
      int numberOfDigits=3, // number of digits 
      int startIndex=0) {  // start index 

      // number of digits must be positive 
      if (numberOfDigits<0) 
       return false; 

      // filenames and their common extension 
      outputFile= filename; 
      extension= ext; 

      // number of digits in the file numbering scheme 
      digits= numberOfDigits; 
      // start numbering at this index 
      currentIndex= startIndex; 

      return true; 
     } 

     // set the callback function that will be called for each frame 
     void setFrameProcessor(void (*frameProcessingCallback)(cv::Mat&, cv::Mat&)) { 

      // invalidate frame processor class instance 
      frameProcessor= 0; 
      // this is the frame processor function that will be called 
      process= frameProcessingCallback; 
      callProcess(); 
     } 

     // set the instance of the class that implements the FrameProcessor interface 
     void setFrameProcessor(FrameProcessor* frameProcessorPtr) { 

      // invalidate callback function 
      process= 0; 
      // this is the frame processor instance that will be called 
      frameProcessor= frameProcessorPtr; 
      callProcess(); 
     } 

     // stop streaming at this frame number 
     void stopAtFrameNo(long frame) { 

      frameToStop= frame; 
     } 

     // process callback to be called 
     void callProcess() { 

      callIt= true; 
     } 

     // do not call process callback 
     void dontCallProcess() { 

      callIt= false; 
     } 

     // to display the processed frames 
     void displayInput(std::string wn) { 

      windowNameInput= wn; 
      //cv::namedWindow(windowNameInput); 
     } 

     // to display the processed frames 
     void displayOutput(std::string wn) { 

      windowNameOutput= wn; 
      //cv::namedWindow(windowNameOutput); 
     } 

     // do not display the processed frames 
     void dontDisplay() { 

      cv::destroyWindow(windowNameInput); 
      cv::destroyWindow(windowNameOutput); 
      windowNameInput.clear(); 
      windowNameOutput.clear(); 
     } 

     // set a delay between each frame 
     // 0 means wait at each frame 
     // negative means no delay 
     void setDelay(int d) { 

      delay= d; 
     } 

     // a count is kept of the processed frames 
     long getNumberOfProcessedFrames() { 

      return fnumber; 
     } 

     // return the size of the video frame 
     cv::Size getFrameSize() { 

     if (images.size()==0) { 

      // get size of from the capture device 
      int w= static_cast<int>(capture.get(CV_CAP_PROP_FRAME_WIDTH)); 
      int h= static_cast<int>(capture.get(CV_CAP_PROP_FRAME_HEIGHT)); 

      return cv::Size(w,h); 

     } else { // if input is vector of images 

      cv::Mat tmp= cv::imread(images[0]); 
      if (!tmp.data) return cv::Size(0,0); 
      else return tmp.size(); 
     } 
     } 

     // return the frame number of the next frame 
     long getFrameNumber() { 

     if (images.size()==0) { 

      // get info of from the capture device 
      long f= static_cast<long>(capture.get(CV_CAP_PROP_POS_FRAMES)); 
      return f; 

     } else { // if input is vector of images 

      return static_cast<long>(itImg-images.begin()); 
     } 
     } 

     // return the position in ms 
     double getPositionMS() { 

      // undefined for vector of images 
      if (images.size()!=0) return 0.0; 

      double t= capture.get(CV_CAP_PROP_POS_MSEC); 
      return t; 
     } 

     // return the frame rate 
     double getFrameRate() { 

      // undefined for vector of images 
      if (images.size()!=0) return 0; 

      double r= capture.get(CV_CAP_PROP_FPS); 
      return r; 
     } 

     // return the number of frames in video 
     long getTotalFrameCount() { 

      // for vector of images 
      if (images.size()!=0) return images.size(); 

      long t= capture.get(CV_CAP_PROP_FRAME_COUNT); 
      return t; 
     } 

     // get the codec of input video 
     int getCodec(char codec[4]) { 

      // undefined for vector of images 
      if (images.size()!=0) return -1; 

      union { 
       int value; 
       char code[4]; } returned; 

      returned.value= static_cast<int>(capture.get(CV_CAP_PROP_FOURCC)); 

      codec[0]= returned.code[0]; 
      codec[1]= returned.code[1]; 
      codec[2]= returned.code[2]; 
      codec[3]= returned.code[3]; 

      return returned.value; 
     } 

     // go to this frame number 
     bool setFrameNumber(long pos) { 

      // for vector of images 
      if (images.size()!=0) { 

       // move to position in vector 
       itImg= images.begin() + pos; 
       // is it a valid position? 
       if (pos < images.size()) 
        return true; 
       else 
        return false; 

      } else { // if input is a capture device 

      return capture.set(CV_CAP_PROP_POS_FRAMES, pos); 
      } 
     } 

     // go to this position 
     bool setPositionMS(double pos) { 

      // not defined in vector of images 
      if (images.size()!=0) 
       return false; 
      else 
       return capture.set(CV_CAP_PROP_POS_MSEC, pos); 
     } 

     // go to this position expressed in fraction of total film length 
     bool setRelativePosition(double pos) { 

      // for vector of images 
      if (images.size()!=0) { 

       // move to position in vector 
       long posI= static_cast<long>(pos*images.size()+0.5); 
       itImg= images.begin() + posI; 
       // is it a valid position? 
       if (posI < images.size()) 
        return true; 
       else 
        return false; 

      } else { // if input is a capture device 

       return capture.set(CV_CAP_PROP_POS_AVI_RATIO, pos); 
      } 
     } 

     // Stop the processing 
     void stopIt() { 

      stop= true; 
     } 

     // Is the process stopped? 
     bool isStopped() { 

      return stop; 
     } 

     // Is a capture device opened? 
     bool isOpened() { 

      return capture.isOpened() || !images.empty(); 
     } 

     // to grab (and process) the frames of the sequence 
     void run(CVWidget *inputWidget, CVWidget *outputWidget) { 

      // current frame 
      cv::Mat frame; 
      // output frame 
      cv::Mat output; 

      // if no capture device has been set 
      if (!isOpened()) 
       return; 

      stop= false; 

      while (!isStopped()) { 

       // read next frame if any 
       if (!readNextFrame(frame)) 
        break; 

       // display input frame 
       if (windowNameInput.length()!=0) 
       // cv::imshow(windowNameInput,frame); 
       inputWidget->sendImage(&frame); 


       // calling the process function or method 
       if (callIt) { 

       // process the frame 
       if (process) 
        process(frame, output); 
       else if (frameProcessor) 
        frameProcessor->process(frame,output); 
       // increment frame number 
       fnumber++; 

       } else { 

       output= frame; 
       } 

       // write output sequence 
       if (outputFile.length()!=0) 
        writeNextFrame(output); 

       // display output frame 
       if (windowNameOutput.length()!=0) 
        //cv::imshow(windowNameOutput,output); 
        outputWidget->sendImage(&output); 

       // introduce a delay 
       if (delay>=0 && cv::waitKey(delay)>=0) 
       stopIt(); 

       // check if we should stop 
       if (frameToStop>=0 && getFrameNumber()==frameToStop) 
        stopIt(); 
      } 
     } 
}; 

#endif 

Answer 1

兼容。雖然我無法找出具體是什麼原因我的問題是，我終於找到了一個適用於我的解決方案：

正如你所建議的，我擺脫了這個VideoProcessor-Utility類，並通過使用QTimer來實現處理幀 - 在MainWindow中的處理循環延遲每幀之間。

取而代之的是我製作的「ProcessorWidget」的VideoProcessor-Class。這個GUI-Widget提供了所有的輸入控件，用於參數化我到目前爲止已經實現的每個處理器功能。

所有的OpenCV代碼現在在ProcessorWidget類中，它有一個公共槽cv :: Mat進程（cv :: Mat輸入），它只需要輸入幀，然後解析所有的用戶輸入參數GUI並在內部處理正確處理器和參數的選擇。

MainWindow現在構造ViewerWidgets和ProcessorWidget並擁有控件和計時。

處理和顯示幀現在只是：

cvWidgetOut->sendImage(&processor->processFrame(input)); 

如果我想添加更多的OpenCV的特點，我沒有改變我的主窗口或GUI。這些都是在ProcessorWidget-Class中完成的。

Answer 2

的OpenCV的highgui窗口處理它自己的事件循環 - 你不能（容易）與QMainWindows事件循環混用。

簡單的方法是使用的QImage和QPainter的搶在Qt的OpenCV中的圖像和顯示 - QImage的的24位RGB888格式與OpenCV中的CV_8UC3