Android OpenCV VideoCapture ::檢索（＆mat）會導致致命的信號11

Question

所以我想製作一個android應用程序，它會通過OpenCV通過JNI和NDK逐幀打開視頻文件，在每個幀中檢測臉部並顯示他們（最終目標是加入算法，以更好地適應糟糕的移動處理）。

我嘗試的第一種方法是MediaMetadataRetriever沒有結果，它只是返回null。

第二種方法是FFMpegMediaMetadataRetriever，它按預期工作，但速度非常慢（高達2fps;太慢），部分原因是您必須從FFMpegMMR的位圖轉換爲Mat，然後檢測並繪製，然後再轉換回位圖，這是我甚至嘗試的非常穴居人。

我目前正在使用的第三種方法是VideoCapture grab（）和retrieve（）。我爲本地代碼做了一個包裝，主要是通過複製OpenCV示例中的facestector。我也嘗試了read（），它可以將兩者結合起來，但是它也會導致致命的信號11（在OpenCV或Android平臺的某些被遺棄的級別上的分割錯誤）。

這裏的應用程序如何獲得文件的絕對路徑（注意，沒有爲FFMpegMMR工作）：

//The file name, file path and filerawresource are used in getting the path of the video file on device's disk 
String videoFileName="samplemp4"; 
String videoFileType=".mp4"; 
int videoFileRawResourceId=R.raw.samplemp4; 

public String getVideoFilePath() 
{ 
    InputStream is = getResources().openRawResource(videoFileRawResourceId); 
    File sampleDir = getDir(videoFileName, Context.MODE_PRIVATE); 
    File sampleFile= new File(sampleDir, videoFileName+videoFileType); 
    try { 
    FileOutputStream os = new FileOutputStream(sampleFile); 

    byte[] buffer = new byte[4096]; 
    int bytesRead; 
    while ((bytesRead = is.read(buffer)) != -1) { 
     os.write(buffer, 0, bytesRead); 
    } 

     is.close(); 
     os.close(); 
    } catch (IOException e) { 
     // TODO Auto-generated catch block 
     e.printStackTrace(); 
    } 
    File videoDir = getDir(videoFileName, Context.MODE_PRIVATE); 
    File videoFile = new File(videoDir, videoFileName+videoFileType); 
    return videoFile.getAbsolutePath(); 
} 

這是本地方法的Java包裝被稱爲和結果使用：

class AsyncPlay extends AsyncTask<String, Mat, Bitmap> 
     { 

     @Override 
     protected Bitmap doInBackground(String... params) { 
      //for(;play;) 
      { 
       if(play) 
       { 
        if(currentTime==0) 
        { 
         test=new Test(); 
         test.startTime.setToNow(); 
         test.type="Video"; 
         frameGrabber.open(videoFilePath); 
        } 
      //publishProgress(retriever.getFrameAtTime(currentTime*1000+111, 
        //  FFmpegMediaMetadataRetriever.OPTION_CLOSEST)); 


        Mat tmp=new Mat(); 
        //frameGrabber.read(tmp); 
        frameGrabber.grab(); 
        frameGrabber.retrieve(tmp); 
        publishProgress(tmp); 

        currentTime+=111; 
        if(currentTime*1000>=duration*1000) 
        { 
         currentTime=0; 
         test.endTime.setToNow(); 
         tester.publishResult(test); 
         frameGrabber.release(); 
        } 
       } 
       try 
       { 
        Thread.sleep(111); 
       } catch (InterruptedException e) 
       { 
        // TODO Auto-generated catch block 
        e.printStackTrace(); 
       } 
      } 
      return null; 
     } 
     @Override 
     protected void onProgressUpdate(Mat... values) { 

      //face detection logic goes here 
      //Bitmap bmp=values[0]; 

      //Mat rgbMat = new Mat(); 
      //Utils.bitmapToMat(bmp, rgbMat); 

      DetectionResult detectionResult = openCVFaceDetector.detectFromImage(values[0], Imgproc.COLOR_RGB2GRAY);//Detecting with a native detector from OpenCV 

      test.addDetection(detectionResult.detection); 

      imageViewOriginal.setImageBitmap(detectionResult.detectedImage); 

      super.onProgressUpdate(values); 
     } 

    };` 

其中FrameGrabber frameGrabber是本機的java包裝器，並且在調用retrieve（）或read（）時發送信號，請注意構造函數open（）和grab（）都工作（或者至少不崩潰應用程序）。

這是包裝（我已經正確通過VideoCapture對象，這是我的教訓是致命的信號的原因之一）：

import org.opencv.core.Mat; 

public class FrameGrabber 
{ 
    private long mNativeObj = 0; 
    private static native long nativeCreateObject(String fileName); 
    private static native void nativeDestroyObject(long thiz); 
    private static native boolean nativeOpen(long thiz, String fileName); 
    private static native boolean nativeGrab(long thiz); 
    private static native boolean nativeRetrieve(long thiz, Mat imageMat); 
    private static native boolean nativeRead(long thiz, Mat imageMat); 
    private static native void nativeRelease(long thiz);  
    public FrameGrabber(String fileName) { 
     mNativeObj = nativeCreateObject(fileName); 
    } 
    public void destroy() { 
     nativeDestroyObject(mNativeObj); 
     mNativeObj = 0; 
    } 
    public boolean open(String fileName) 
    { 
     return nativeOpen(mNativeObj, fileName); 
    } 
    public boolean grab() 
    { 
     return nativeGrab(mNativeObj); 
    } 
    public boolean retrieve(Mat imageMat) 
    { 
     return nativeRetrieve(mNativeObj, imageMat); 
    }  
    public boolean read(Mat imageMat) 
    { 
     return nativeRead(mNativeObj, imageMat); 
    }  
    public void release() 
    { 
     nativeRelease(mNativeObj); 
    } 
} 

，這是它的本地部分（確切的路線造成的誤差是result = ((VideoCapture*)thiz)->retrieve((*((Mat*)imageMat)));，其是在該方法中nativeRetrieve（）

#include <string> 
#include <vector> 
#include <opencv2/core/core.hpp> 
#include <opencv2/highgui/highgui.hpp> 
#include <FrameGrabber_jni.h> 

#include <android/log.h> 

#define LOG_TAG "FrameGrabber" 
#define LOGD(...) ((void)__android_log_print(ANDROID_LOG_DEBUG, LOG_TAG, __VA_ARGS__)) 

using namespace std; 
using namespace cv; 

inline void jStringToString(JNIEnv * jenv, jstring jString, string stdName) 
{ 
    LOGD("Java_boris_springar_diplomska_FrameGrabber_jStringToString enter"); 
    LOGD("grabber is Making jnamestr");//this is buggy as hell, the line serves as a debug tool 
    const char* jnamestr = jenv->GetStringUTFChars(jString, NULL); 
    string stdNameTmp(jnamestr); 
    stdName=stdNameTmp; 
    LOGD("grabber is releasing jnamestr"); 
    jenv->ReleaseStringUTFChars(jString, jnamestr); 
} 

/* 
* CONSTRUCTOR AND DESTRUCTOR 
*/ 
JNIEXPORT jlong JNICALL Java_boris_springar_diplomska_FrameGrabber_nativeCreateObject 
(JNIEnv * jenv, jclass, jstring jFileName) 
{ 
    LOGD("Java_boris_springar_diplomska_FrameGrabber_nativeCreateObject enter"); 
    string stdFileName; 
    jStringToString(jenv, jFileName,stdFileName); 
    jlong result = 0; 

    try 
    { 
     result = (jlong)new VideoCapture(stdFileName); 
    } 
    catch(cv::Exception& e) 
    { 
     LOGD("nativeCreateObject caught cv::Exception: %s", e.what()); 
     jclass je = jenv->FindClass("org/opencv/core/CvException"); 
     if(!je) 
      je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, e.what()); 
    } 
    catch (...) 
    { 
     LOGD("nativeCreateObject caught unknown exception"); 
     jclass je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, "Unknown exception in JNI code of FrameGrabber.nativeCreateObject()"); 
     return 0; 
    } 

    LOGD("Java_boris_springar_diplomska_FrameGrabber_nativeCreateObject exit"); 
    return result; 
} 

//should work 
JNIEXPORT void JNICALL Java_boris_springar_diplomska_FrameGrabber_nativeDestroyObject 
(JNIEnv * jenv, jclass, jlong thiz) 
{ 
    LOGD("Java_boris_springar_diplomska_FrameGrabber_nativeDestroyObject enter"); 
    try 
    { 
     if(thiz != 0) 
     { 
      delete (VideoCapture*)thiz; 
     } 
    } 
    catch(cv::Exception& e) 
    { 
     LOGD("nativeestroyObject caught cv::Exception: %s", e.what()); 
     jclass je = jenv->FindClass("org/opencv/core/CvException"); 
     if(!je) 
      je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, e.what()); 
    } 
    catch (...) 
    { 
     LOGD("nativeDestroyObject caught unknown exception"); 
     jclass je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, "Unknown exception in JNI code of FrameGrabber.nativeDestroyObject()"); 
    } 
    LOGD("Java_boris_springar_diplomska_FrameGrabber_nativeDestroyObject exit"); 
} 


/* 
* CORE METHODS 
*/ 

//Open function opens the filename for playing 
JNIEXPORT jboolean JNICALL Java_boris_springar_diplomska_FrameGrabber_nativeOpen 
(JNIEnv * jenv, jclass,jlong thiz,jstring jFileName) 
{ 
    LOGD("Java_boris_springar_diplomska_FrameGrabber_open enter"); 
    string stdFileName; 
    jStringToString(jenv, jFileName,stdFileName); 

    jboolean result = false; 

    try 
    { 
     result = ((VideoCapture*)thiz)->open(stdFileName); 
    } 
    catch(cv::Exception& e) 
    { 
     LOGD("frame grabber open exception caught cv::Exception: %s", e.what()); 
     jclass je = jenv->FindClass("org/opencv/core/CvException"); 
     if(!je) 
      je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, e.what()); 
    } 
    catch (...) 
    { 
     LOGD("frame grabber open caught unknown exception"); 
     jclass je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, "Unknown exception in JNI code of FrameGrabber.open()"); 
     return 0; 
    } 

    LOGD("Java_boris_springar_diplomska_FrameGrabber_open exit"); 

    return result; 
} 

//grab grabs the next frame from file or camera 
JNIEXPORT jboolean JNICALL Java_boris_springar_diplomska_FrameGrabber_nativeGrab 
(JNIEnv * jenv, jclass,jlong thiz) 
{ 
    LOGD("Java_boris_springar_diplomska_FrameGrabber_grab enter"); 
    jboolean result = false; 

    try 
    { 
     result = ((VideoCapture*)thiz)->grab(); 
    } 
    catch(cv::Exception& e) 
    { 
     LOGD("frame grabber grab exception caught cv::Exception: %s", e.what()); 
     jclass je = jenv->FindClass("org/opencv/core/CvException"); 
     if(!je) 
      je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, e.what()); 
    } 
    catch (...) 
    { 
     LOGD("frame grabber grab caught unknown exception"); 
     jclass je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, "Unknown exception in JNI code of FrameGrabber.grab()"); 
     return 0; 
    } 

    LOGD("Java_boris_springar_diplomska_FrameGrabber_grab exit"); 

    return result; 
} 

//retrieve retrieves the next frame and writes it to the image matrix 
JNIEXPORT jboolean JNICALL Java_boris_springar_diplomska_FrameGrabber_nativeRetrieve 
(JNIEnv * jenv, jclass,jlong thiz, jlong imageMat) 
{ 
    LOGD("Java_boris_springar_diplomska_FrameGrabber_retrieve enter"); 
    jboolean result = false; 

    try 
    { 
     LOGD("grabber trying to retrieve"); 
     result = ((VideoCapture*)thiz)->retrieve((*((Mat*)imageMat)));//should write the current frame to the image matrix 
    } 
    catch(cv::Exception& e) 
    { 
     LOGD("frame grabber retrieve exception caught cv::Exception: %s", e.what()); 
     jclass je = jenv->FindClass("org/opencv/core/CvException"); 
     if(!je) 
      je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, e.what()); 
    } 
    catch (...) 
    { 
     LOGD("frame grabber retrieve caught unknown exception"); 
     jclass je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, "Unknown exception in JNI code of FrameGrabber.retrieve(fileName)"); 
     return 0; 
    } 

    LOGD("Java_boris_springar_diplomska_FrameGrabber_retrieve exit"); 

    return result; 
} 

//read combines grab and retrieve and writes the stuff to the image matrix 
JNIEXPORT jboolean JNICALL Java_boris_springar_diplomska_FrameGrabber_nativeRead 
(JNIEnv * jenv, jclass,jlong thiz, jlong imageMat) 
{ 
    LOGD("Java_boris_springar_diplomska_FrameGrabber_read enter"); 
    LOGD("grabber setting result to false"); 
    jboolean result = false; 

    try 
    { 
     LOGD("grabber trying to read capture"); 
     result = ((VideoCapture*)thiz)->read((*((Mat*)imageMat)));//should write the current frame to the image matrix 
    } 
    catch(cv::Exception& e) 
    { 
     LOGD("frame grabber read exception caught cv::Exception: %s", e.what()); 
     jclass je = jenv->FindClass("org/opencv/core/CvException"); 
     if(!je) 
      je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, e.what()); 
    } 
    catch (...) 
    { 
     LOGD("frame grabber read caught unknown exception"); 
     jclass je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, "Unknown exception in JNI code of FrameGrabber.open(fileName)"); 
     return 0; 
    } 

    LOGD("Java_boris_springar_diplomska_FrameGrabber_read exit"); 

    return result; 
} 

//Release releases the resource it's using, I hope 
JNIEXPORT void JNICALL Java_boris_springar_diplomska_FrameGrabber_nativeRelease 
(JNIEnv * jenv, jclass,jlong thiz) 
{ 
    LOGD("Java_boris_springar_diplomska_FrameGrabber_release enter"); 
    jboolean result = false; 

    try 
    { 
     ((VideoCapture*)thiz)->release();//should release 
    } 
    catch(cv::Exception& e) 
    { 
     LOGD("frame grabber read exception caught cv::Exception: %s", e.what()); 
     jclass je = jenv->FindClass("org/opencv/core/CvException"); 
     if(!je) 
      je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, e.what()); 
    } 
    catch (...) 
    { 
     LOGD("frame grabber release caught unknown exception"); 
     jclass je = jenv->FindClass("java/lang/Exception"); 
     jenv->ThrowNew(je, "Unknown exception in JNI code of FrameGrabber.open(fileName)"); 
    } 

    LOGD("Java_boris_springar_diplomska_FrameGrabber_release exit"); 
} 

同時，也是logcat的輸出：

07-22 17:36:24.886: D/FrameGrabber(15359): Java_boris_springar_diplomska_FrameGrabber_nativeCreateObject enter 
07-22 17:36:24.886: D/FrameGrabber(15359): Java_boris_springar_diplomska_FrameGrabber_jStringToString enter 
07-22 17:36:24.886: D/FrameGrabber(15359): grabber is Making jnamestr 
07-22 17:36:24.886: D/FrameGrabber(15359): grabber is releasing jnamestr 
07-22 17:36:24.886: D/FrameGrabber(15359): Java_boris_springar_diplomska_FrameGrabber_nativeCreateObject exit 
07-22 17:36:24.936: I/dalvikvm(15359): threadid=3: reacting to signal 3 
07-22 17:36:24.936: I/dalvikvm(15359): Wrote stack traces to '/data/anr/traces.txt' 
07-22 17:36:24.946: D/libEGL(15359): loaded /system/lib/egl/libEGL_mali.so 
07-22 17:36:24.956: D/libEGL(15359): loaded /system/lib/egl/libGLESv1_CM_mali.so 
07-22 17:36:24.976: D/libEGL(15359): loaded /system/lib/egl/libGLESv2_mali.so 
07-22 17:36:25.006: D/OpenGLRenderer(15359): Enabling debug mode 0 
07-22 17:36:27.336: D/FrameGrabber(15359): Java_boris_springar_diplomska_FrameGrabber_open enter 
07-22 17:36:27.336: D/FrameGrabber(15359): Java_boris_springar_diplomska_FrameGrabber_jStringToString enter 
07-22 17:36:27.336: D/FrameGrabber(15359): grabber is Making jnamestr 
07-22 17:36:27.336: D/FrameGrabber(15359): grabber is releasing jnamestr 
07-22 17:36:27.336: D/FrameGrabber(15359): Java_boris_springar_diplomska_FrameGrabber_open exit 
07-22 17:36:27.336: D/FrameGrabber(15359): Java_boris_springar_diplomska_FrameGrabber_grab enter 
07-22 17:36:27.336: D/FrameGrabber(15359): Java_boris_springar_diplomska_FrameGrabber_grab exit 
07-22 17:36:27.336: D/FrameGrabber(15359): Java_boris_springar_diplomska_FrameGrabber_retrieve enter 
07-22 17:36:27.336: D/FrameGrabber(15359): grabber trying to retrieve 
07-22 17:36:27.336: A/libc(15359): Fatal signal 11 (SIGSEGV) at 0x1d400019 (code=1) 

調試此代碼的問題在於它是Eclipse中的JNI，我不知道如何在Eclipse中甚至進行調試C++工作，這就是爲什麼我使用日誌消息來查找代碼中的錯誤。那些幫助非常大，因爲open（）和grab（）不起作用，我發現並壓扁了一個bug，在那裏我忘記了將thiz傳遞給方法。 retrieve（）的問題在於我不能在我的生活中找到它的來源。在highgui.hpp中有部分定義，但沒有實現，我可以在這裏放置日誌消息來幫助我調試。

可能的解決方案是：這是由檢索（）的支持，但我不知道哪些文件格式，可能是

使用非絕對文件路徑

• 不同的文件格式？儘管兩人的open（）和抓鬥（）並沒有造成致命的信號
• 報廢整個事情並使用PNG（我真的想雖然這出）

所以，如果有人能告訴我在哪兒可以找到retrieve（）和read（）的實現，我非常感謝。或者，如果這是我錯過的明顯和愚蠢的東西（我希望）。

同時，我會嘗試找到一種在Eclipse中調試C++的方法，也許嘗試另一種格式？

Answer 1

發現了它，最後，在亞歷山大的建議，我設法找到從Java傳遞給JNI該對象的類型不匹配。

原來，錯誤發生在聲明本地方法的java部分。我不必傳遞Mat，而必須通過一個很長的Mat.getNativeObjAddr（）。

所以不是

private static native boolean nativeRetrieve(long thiz, Mat imageMat); 
    private static native boolean nativeRead(long thiz, Mat imageMat); 
    public boolean retrieve(Mat imageMat) 
    { 
     return nativeRetrieve(mNativeObj, imageMat); 
    }  
    public boolean read(Mat imageMat) 
    { 
     return nativeRead(mNativeObj, imageMat); 
    }  

我用這個：

 private static native boolean nativeRetrieve(long thiz, long imageMat); 
     private static native boolean nativeRead(long thiz, long imageMat); 
     public boolean retrieve(Mat imageMat) 
     { 
      return nativeRetrieve(mNativeObj, imageMat.getNativeObjAddr()); 
     }  
     public boolean read(Mat imageMat) 
     { 
      return nativeRead(mNativeObj, imageMat.getNativeObjAddr()); 
     } 

注：傳遞錯誤的類型並沒有引發C++異常，即使代碼在TRY ... CATCH 。 try ... catch可能寫錯了，我不知道。