2012-10-18 33 views
17

我想爲使用OpenCV的一些C++代碼編寫一個python包裝,但是我很難將返回OpenCV C++ Mat對象的結果返回給python解釋器。爲使用OpenCV的C++代碼編寫Python綁定

我看了OpenCV的源代碼,發現了文件cv2.cpp,它具有轉換函數來執行PyObject *和OpenCV的Mat之間的來回轉換。我使用了這些轉換函數,但在嘗試使用它們時出現了分段錯誤。

我基本上需要一些關於如何連接使用OpenCV的Python和C++代碼的建議/示例代碼/在線參考,特別是能夠將OpenCV的C++ Mat返回給python解釋器,或者提供關於如何/向何處開始調查分段故障的原因。

目前我正在使用Boost Python來包裝代碼。

在此先感謝您的答覆。

相關的代碼:從OpenCV中的源採取

// This is the function that is giving the segmentation fault. 
PyObject* ABC::doSomething(PyObject* image) 
{ 
    Mat m; 
    pyopencv_to(image, m); // This line gives segmentation fault. 

    // Some code to create cppObj from CPP library that uses OpenCV 
    cv::Mat processedImage = cppObj->align(m); 

    return pyopencv_from(processedImage); 
} 

的轉換函數如下。轉換代碼在註釋行中使用「if(!PyArray_Check(o))...」提供分段錯誤。

static int pyopencv_to(const PyObject* o, Mat& m, const char* name = "<unknown>", bool allowND=true) 
{ 
    if(!o || o == Py_None) 
    { 
     if(!m.data) 
      m.allocator = &g_numpyAllocator; 
     return true; 
    } 

    if(!PyArray_Check(o)) // Segmentation fault inside PyArray_Check(o) 
    { 
     failmsg("%s is not a numpy array", name); 
     return false; 
    } 

    int typenum = PyArray_TYPE(o); 
    int type = typenum == NPY_UBYTE ? CV_8U : typenum == NPY_BYTE ? CV_8S : 
       typenum == NPY_USHORT ? CV_16U : typenum == NPY_SHORT ? CV_16S : 
       typenum == NPY_INT || typenum == NPY_LONG ? CV_32S : 
       typenum == NPY_FLOAT ? CV_32F : 
       typenum == NPY_DOUBLE ? CV_64F : -1; 

    if(type < 0) 
    { 
     failmsg("%s data type = %d is not supported", name, typenum); 
     return false; 
    } 

    int ndims = PyArray_NDIM(o); 
    if(ndims >= CV_MAX_DIM) 
    { 
     failmsg("%s dimensionality (=%d) is too high", name, ndims); 
     return false; 
    } 

    int size[CV_MAX_DIM+1]; 
    size_t step[CV_MAX_DIM+1], elemsize = CV_ELEM_SIZE1(type); 
    const npy_intp* _sizes = PyArray_DIMS(o); 
    const npy_intp* _strides = PyArray_STRIDES(o); 
    bool transposed = false; 

    for(int i = 0; i < ndims; i++) 
    { 
     size[i] = (int)_sizes[i]; 
     step[i] = (size_t)_strides[i]; 
    } 

    if(ndims == 0 || step[ndims-1] > elemsize) { 
     size[ndims] = 1; 
     step[ndims] = elemsize; 
     ndims++; 
    } 

    if(ndims >= 2 && step[0] < step[1]) 
    { 
     std::swap(size[0], size[1]); 
     std::swap(step[0], step[1]); 
     transposed = true; 
    } 

    if(ndims == 3 && size[2] <= CV_CN_MAX && step[1] == elemsize*size[2]) 
    { 
     ndims--; 
     type |= CV_MAKETYPE(0, size[2]); 
    } 

    if(ndims > 2 && !allowND) 
    { 
     failmsg("%s has more than 2 dimensions", name); 
     return false; 
    } 

    m = Mat(ndims, size, type, PyArray_DATA(o), step); 

    if(m.data) 
    { 
     m.refcount = refcountFromPyObject(o); 
     m.addref(); // protect the original numpy array from deallocation 
        // (since Mat destructor will decrement the reference counter) 
    }; 
    m.allocator = &g_numpyAllocator; 

    if(transposed) 
    { 
     Mat tmp; 
     tmp.allocator = &g_numpyAllocator; 
     transpose(m, tmp); 
     m = tmp; 
    } 
    return true; 
} 

static PyObject* pyopencv_from(const Mat& m) 
{ 
    if(!m.data) 
     Py_RETURN_NONE; 
    Mat temp, *p = (Mat*)&m; 
    if(!p->refcount || p->allocator != &g_numpyAllocator) 
    { 
     temp.allocator = &g_numpyAllocator; 
     m.copyTo(temp); 
     p = &temp; 
    } 
    p->addref(); 
    return pyObjectFromRefcount(p->refcount); 
} 

我的Python測試程序:

import pysomemodule # My python wrapped library. 
import cv2 

def main(): 
    myobj = pysomemodule.ABC("faces.train") # Create python object. This works. 
    image = cv2.imread('61.jpg') 
    processedImage = myobj.doSomething(image) 
    cv2.imshow("test", processedImage) 
    cv2.waitKey() 

if __name__ == "__main__": 
    main() 

回答

29

我解決了這個問題,所以我想我會跟其他人誰可能有同樣的問題在這裏分享。

基本上,爲了擺脫分段錯誤,我需要調用numpy的import_array()函數。

「高級別」的觀點從蟒蛇運行C++代碼是這樣的:

假設你有Python中的函數foo(arg)是一些C++函數的綁定。當您撥打foo(myObj)時,必須有一些代碼將python對象「myObj」轉換爲您的C++代碼可以執行的形式。此代碼通常是使用諸如SWIG或Boost :: Python之類的工具半自動創建的。 (我在下面的例子中使用了Boost :: Python。)

現在,foo(arg)是一些用於某些C++函數的python綁定。這個C++函數將收到一個通用的PyObject指針作爲參數。您需要有C++代碼才能將此PyObject指針轉換爲「等效」C++對象。在我的情況下,我的python代碼爲OpenCV圖像傳遞了一個OpenCV numpy數組作爲函數的參數。 C++中的「等價」形式是一個OpenCV C++ Mat對象。 OpenCV在cv2.cpp中提供了一些代碼(轉載如下),將PyObject指針(代表numpy數組)轉換爲C++ Mat。簡單的數據類型,如int和string,不需要用戶編寫這些轉換函數,因爲它們是由Boost :: Python自動轉換的。

PyObject指針轉換爲合適的C++表單後,C++代碼可以對其執行操作。當數據必須從C++返回到python時,需要使用C++代碼將數據的C++表示形式轉換爲某種形式的PyObject,這時會出現類似的情況。 Boost :: Python將負責將PyObject轉換爲相應的Python表單。當foo(arg)在python中返回結果時,它的格式是python可用的。而已。

下面的代碼展示瞭如何包裝一個C++類「ABC」並暴露它的方法「doSomething」,它接受來自python的numpy數組(對於圖像),將其轉換爲OpenCV的C++ Mat,做一些處理,轉換PyObject *的結果,並將其返回給python解釋器。您可以公開許多您希望的功能/方法(請參閱下面的代碼中的註釋)。

abc.hpp:

#ifndef ABC_HPP 
#define ABC_HPP 

#include <Python.h> 
#include <string> 

class ABC 
{ 
    // Other declarations 
    ABC(); 
    ABC(const std::string& someConfigFile); 
    virtual ~ABC(); 
    PyObject* doSomething(PyObject* image); // We want our python code to be able to call this function to do some processing using OpenCV and return the result. 
    // Other declarations 
}; 

#endif 

abc.cpp:

#include "abc.hpp" 
#include "my_cpp_library.h" // This is what we want to make available in python. It uses OpenCV to perform some processing. 

#include "numpy/ndarrayobject.h" 
#include "opencv2/core/core.hpp" 

// The following conversion functions are taken from OpenCV's cv2.cpp file inside modules/python/src2 folder. 
static PyObject* opencv_error = 0; 

static int failmsg(const char *fmt, ...) 
{ 
    char str[1000]; 

    va_list ap; 
    va_start(ap, fmt); 
    vsnprintf(str, sizeof(str), fmt, ap); 
    va_end(ap); 

    PyErr_SetString(PyExc_TypeError, str); 
    return 0; 
} 

class PyAllowThreads 
{ 
public: 
    PyAllowThreads() : _state(PyEval_SaveThread()) {} 
    ~PyAllowThreads() 
    { 
     PyEval_RestoreThread(_state); 
    } 
private: 
    PyThreadState* _state; 
}; 

class PyEnsureGIL 
{ 
public: 
    PyEnsureGIL() : _state(PyGILState_Ensure()) {} 
    ~PyEnsureGIL() 
    { 
     PyGILState_Release(_state); 
    } 
private: 
    PyGILState_STATE _state; 
}; 

#define ERRWRAP2(expr) \ 
try \ 
{ \ 
    PyAllowThreads allowThreads; \ 
    expr; \ 
} \ 
catch (const cv::Exception &e) \ 
{ \ 
    PyErr_SetString(opencv_error, e.what()); \ 
    return 0; \ 
} 

using namespace cv; 

static PyObject* failmsgp(const char *fmt, ...) 
{ 
    char str[1000]; 

    va_list ap; 
    va_start(ap, fmt); 
    vsnprintf(str, sizeof(str), fmt, ap); 
    va_end(ap); 

    PyErr_SetString(PyExc_TypeError, str); 
    return 0; 
} 

static size_t REFCOUNT_OFFSET = (size_t)&(((PyObject*)0)->ob_refcnt) + 
    (0x12345678 != *(const size_t*)"\x78\x56\x34\x12\0\0\0\0\0")*sizeof(int); 

static inline PyObject* pyObjectFromRefcount(const int* refcount) 
{ 
    return (PyObject*)((size_t)refcount - REFCOUNT_OFFSET); 
} 

static inline int* refcountFromPyObject(const PyObject* obj) 
{ 
    return (int*)((size_t)obj + REFCOUNT_OFFSET); 
} 

class NumpyAllocator : public MatAllocator 
{ 
public: 
    NumpyAllocator() {} 
    ~NumpyAllocator() {} 

    void allocate(int dims, const int* sizes, int type, int*& refcount, 
        uchar*& datastart, uchar*& data, size_t* step) 
    { 
     PyEnsureGIL gil; 

     int depth = CV_MAT_DEPTH(type); 
     int cn = CV_MAT_CN(type); 
     const int f = (int)(sizeof(size_t)/8); 
     int typenum = depth == CV_8U ? NPY_UBYTE : depth == CV_8S ? NPY_BYTE : 
         depth == CV_16U ? NPY_USHORT : depth == CV_16S ? NPY_SHORT : 
         depth == CV_32S ? NPY_INT : depth == CV_32F ? NPY_FLOAT : 
         depth == CV_64F ? NPY_DOUBLE : f*NPY_ULONGLONG + (f^1)*NPY_UINT; 
     int i; 
     npy_intp _sizes[CV_MAX_DIM+1]; 
     for(i = 0; i < dims; i++) 
     { 
      _sizes[i] = sizes[i]; 
     } 

     if(cn > 1) 
     { 
      /*if(_sizes[dims-1] == 1) 
       _sizes[dims-1] = cn; 
      else*/ 
       _sizes[dims++] = cn; 
     } 

     PyObject* o = PyArray_SimpleNew(dims, _sizes, typenum); 

     if(!o) 
     { 
      CV_Error_(CV_StsError, ("The numpy array of typenum=%d, ndims=%d can not be created", typenum, dims)); 
     } 
     refcount = refcountFromPyObject(o); 

     npy_intp* _strides = PyArray_STRIDES(o); 
     for(i = 0; i < dims - (cn > 1); i++) 
      step[i] = (size_t)_strides[i]; 
     datastart = data = (uchar*)PyArray_DATA(o); 
    } 

    void deallocate(int* refcount, uchar*, uchar*) 
    { 
     PyEnsureGIL gil; 
     if(!refcount) 
      return; 
     PyObject* o = pyObjectFromRefcount(refcount); 
     Py_INCREF(o); 
     Py_DECREF(o); 
    } 
}; 

NumpyAllocator g_numpyAllocator; 

enum { ARG_NONE = 0, ARG_MAT = 1, ARG_SCALAR = 2 }; 

static int pyopencv_to(const PyObject* o, Mat& m, const char* name = "<unknown>", bool allowND=true) 
{ 
    //NumpyAllocator g_numpyAllocator; 
    if(!o || o == Py_None) 
    { 
     if(!m.data) 
      m.allocator = &g_numpyAllocator; 
     return true; 
    } 

    if(!PyArray_Check(o)) 
    { 
     failmsg("%s is not a numpy array", name); 
     return false; 
    } 

    int typenum = PyArray_TYPE(o); 
    int type = typenum == NPY_UBYTE ? CV_8U : typenum == NPY_BYTE ? CV_8S : 
       typenum == NPY_USHORT ? CV_16U : typenum == NPY_SHORT ? CV_16S : 
       typenum == NPY_INT || typenum == NPY_LONG ? CV_32S : 
       typenum == NPY_FLOAT ? CV_32F : 
       typenum == NPY_DOUBLE ? CV_64F : -1; 

    if(type < 0) 
    { 
     failmsg("%s data type = %d is not supported", name, typenum); 
     return false; 
    } 

    int ndims = PyArray_NDIM(o); 
    if(ndims >= CV_MAX_DIM) 
    { 
     failmsg("%s dimensionality (=%d) is too high", name, ndims); 
     return false; 
    } 

    int size[CV_MAX_DIM+1]; 
    size_t step[CV_MAX_DIM+1], elemsize = CV_ELEM_SIZE1(type); 
    const npy_intp* _sizes = PyArray_DIMS(o); 
    const npy_intp* _strides = PyArray_STRIDES(o); 
    bool transposed = false; 

    for(int i = 0; i < ndims; i++) 
    { 
     size[i] = (int)_sizes[i]; 
     step[i] = (size_t)_strides[i]; 
    } 

    if(ndims == 0 || step[ndims-1] > elemsize) { 
     size[ndims] = 1; 
     step[ndims] = elemsize; 
     ndims++; 
    } 

    if(ndims >= 2 && step[0] < step[1]) 
    { 
     std::swap(size[0], size[1]); 
     std::swap(step[0], step[1]); 
     transposed = true; 
    } 

    if(ndims == 3 && size[2] <= CV_CN_MAX && step[1] == elemsize*size[2]) 
    { 
     ndims--; 
     type |= CV_MAKETYPE(0, size[2]); 
    } 

    if(ndims > 2 && !allowND) 
    { 
     failmsg("%s has more than 2 dimensions", name); 
     return false; 
    } 

    m = Mat(ndims, size, type, PyArray_DATA(o), step); 

    if(m.data) 
    { 
     m.refcount = refcountFromPyObject(o); 
     m.addref(); // protect the original numpy array from deallocation 
        // (since Mat destructor will decrement the reference counter) 
    }; 
    m.allocator = &g_numpyAllocator; 

    if(transposed) 
    { 
     Mat tmp; 
     tmp.allocator = &g_numpyAllocator; 
     transpose(m, tmp); 
     m = tmp; 
    } 
    return true; 
} 

static PyObject* pyopencv_from(const Mat& m) 
{ 
    if(!m.data) 
     Py_RETURN_NONE; 
    Mat temp, *p = (Mat*)&m; 
    if(!p->refcount || p->allocator != &g_numpyAllocator) 
    { 
     temp.allocator = &g_numpyAllocator; 
     m.copyTo(temp); 
     p = &temp; 
    } 
    p->addref(); 
    return pyObjectFromRefcount(p->refcount); 
} 

ABC::ABC() {} 
ABC::~ABC() {} 
// Note the import_array() from NumPy must be called else you will experience segmentation faults. 
ABC::ABC(const std::string &someConfigFile) 
{ 
    // Initialization code. Possibly store someConfigFile etc. 
    import_array(); // This is a function from NumPy that MUST be called. 
    // Do other stuff 
} 

// The conversions functions above are taken from OpenCV. The following function is 
// what we define to access the C++ code we are interested in. 
PyObject* ABC::doSomething(PyObject* image) 
{ 
    cv::Mat cvImage; 
    pyopencv_to(image, cvImage); // From OpenCV's source 

    MyCPPClass obj; // Some object from the C++ library. 
    cv::Mat processedImage = obj.process(cvImage); 

    return pyopencv_from(processedImage); // From OpenCV's source 
} 

的代碼來使用升壓Python創建蟒模塊。我把這個和下面的Makefile從http://jayrambhia.wordpress.com/tag/boost/

pysomemodule.cpp:

#include <string>  
#include<boost/python.hpp> 
#include "abc.hpp" 

using namespace boost::python; 

BOOST_PYTHON_MODULE(pysomemodule) 
{ 
    class_<ABC>("ABC", init<const std::string &>()) 
     .def(init<const std::string &>()) 
     .def("doSomething", &ABC::doSomething) // doSomething is the method in class ABC you wish to expose. One line for each method (or function depending on how you structure your code). Note: You don't have to expose everything in the library, just the ones you wish to make available to python. 
    ; 
} 

最後,在Makefile(編譯成功在Ubuntu,但應以最小的調整可能在其他地方工作)。

PYTHON_VERSION = 2.7 
PYTHON_INCLUDE = /usr/include/python$(PYTHON_VERSION) 

# location of the Boost Python include files and library 
BOOST_INC = /usr/local/include/boost 
BOOST_LIB = /usr/local/lib 

OPENCV_LIB = `pkg-config --libs opencv` 
OPENCV_CFLAGS = `pkg-config --cflags opencv` 

MY_CPP_LIB = lib_my_cpp_library.so 

TARGET = pysomemodule 
SRC = pysomemodule.cpp abc.cpp 
OBJ = pysomemodule.o abc.o 

$(TARGET).so: $(OBJ) 
    g++ -shared $(OBJ) -L$(BOOST_LIB) -lboost_python -L/usr/lib/python$(PYTHON_VERSION)/config -lpython$(PYTHON_VERSION) -o $(TARGET).so $(OPENCV_LIB) $(MY_CPP_LIB) 

$(OBJ): $(SRC) 
    g++ -I$(PYTHON_INCLUDE) -I$(BOOST_INC) $(OPENCV_CFLAGS) -fPIC -c $(SRC) 

clean: 
    rm -f $(OBJ) 
    rm -f $(TARGET).so 

成功編譯庫後,應該在目錄中有一個文件「pysomemodule.so」。把這個lib文件放在python解釋器可以訪問的地方。然後,您可以導入該模塊並創建類「ABC」的一個實例,上面如下:

import pysomemodule 

foo = pysomemodule.ABC("config.txt") # This will create an instance of ABC 

現在,給定的OpenCV numpy的陣列圖像時,我們可以使用調用C++函數:

processedImage = foo.doSomething(image) # Where the argument "image" is a OpenCV numpy image. 

請注意,您將需要Boost Python,Numpy dev以及Python開發庫來創建綁定。

以下兩個鏈接中的NumPy文檔在幫助理解轉換代碼中使用的方法以及爲什麼必須調用import_array()方面特別有用。特別是官方的numpy文檔有助於理解OpenCV的Python綁定代碼。

http://dsnra.jpl.nasa.gov/software/Python/numpydoc/numpy-13.html http://docs.scipy.org/doc/numpy/user/c-info.how-to-extend.html

希望這有助於。

+0

嗨,lightalchemist,感謝您張貼您的解決方案。我已經使用OpenCV 2.4.3(從cv2.cpp獲取pyopencv_to和pyopencv_from函數)以及暴露一個函數的類似解決方案。該模塊在ipython中可以加載,該函數在那裏可見,它可以解析參數,但一旦達到PyEnsureGIL就會崩潰。我已經嘗試過你的解決方案,並且舊的pyopencv_to函數可以工作(它會執行),但是在嘗試輸出時會崩潰。我將發佈一個單獨的問題,並在短時間內爲您提供一個鏈接,以防您發現問題出在哪裏。 –

+0

這裏是我的問題的鏈接:http://stackoverflow.com/questions/13745265/exposing-opencv-based-c-function-with-mat-numpy-conversion-to-python –

5

我希望這可以幫助人們尋找一種快速簡便的方法。

這裏是github repo與我寫的開放C++代碼,用盡可能少的痛苦暴露代碼使用OpenCV的Mat類。

[更新]此代碼現在爲OpenCV的2.XOpenCV的3.X。 CMake和Python 3.X的實驗支持現在也可用。

+1

你爲3.x版本的代碼是大!一開始我以爲它有memleak,但最終發現這個錯誤在我的代碼中;)2.xi版本推薦使用https://github.com/spillai/numpy-opencv-converter-很好代碼與很多類型的轉換器(如Mat,Mat3f,Point2d等 - 上帝保佑模板:))。 – cyriel

+0

我也遇到了使用你的代碼的分段錯誤。在我的Python代碼中添加'import pbcvt' _before_我自己的庫導入最終修復了它。 (這對我來說並不直觀,但爲什麼這是必要的。) –

+0

@ManuCJ,你是對的,這真的很奇怪。 pbcvt實際上是作爲您自己的庫的代碼模板而設計的。因此,我會研究pbcvt的代碼沒有的東西(其中一些定義在頭文件和cpp文件中,也許?)。如果您仍然受到此問題的影響,我建議您使用最少的代碼示例在GitHub上提交它。 –