如何避免使用PortAudio和OpenCV進行不一致的音頻播放？

Question

我正在使用opencv（用於對象識別）與portaudio結合播放基於視頻輸入的聲音。基本上，我的目標是以不同的速率播放一定音高/頻率的正弦波音。它有效，但結果非常不可預測。有時候音頻回放會起作用（程序運行緩慢，但工作起來），有時候不會發生音頻回放。簡而言之，這是我的程序所做的：

開始攝像頭圖像 - >獲取攝像頭圖像 - >在圖像中選擇區域 - >返回視頻輸入 - > while（frame exists） - >跟蹤對象位置 - >初始化端口音頻工具 - >播放基於位置的聲音 - >終止Portaudio工具

我似乎無法弄清爲什麼音頻播放不一致。你們有沒有任何提示？我一直在閱讀，我的想法是，這是一個延遲問題，但我真的沒有經歷過這件事。當我在沒有opencv的情況下使用portaudio時，沒有延遲問題發生，所以我知道它與將這兩者結合起來。任何幫助表示讚賞。

while (frame) 
{ 
    cvCopyImage(frame, drawImg); 

    // process 
    track(frame); 

    // get result 
    CvRect r; 
    float confidence; 
    bool valid; 
    /* getRoi tells us if the region being tracked on the screen 
    * is the same region that we chose prior to entering this while loop 
    */ 
    getRoi(&r, &confidence, &valid); 

    // show 
    cvDrawRect(drawImg, cvPoint(r.x, r.y), 
     cvPoint(r.x + r.width - 1, r.y + r.height - 1), 
     valid ? cvScalar(0, 255, 0) : cvScalar(0, 255, 255), 
     2 
    ); 
    writeLogo(drawImg,"USC-IRIS"); 
    int xpos = r.x; 
    int ypos = r.y; 



    cvShowImage("Tracking", drawImg); 
    cout << "valid " << valid << endl; 
    cout << "conf val " << confidence << endl; 
    cout << "xpos, ypos " << xpos << ", " << ypos << endl; 
      //If the region on the screen is the region we chose 
      //then we should play specific sounds 
    if(valid){ 

     sI->soundWrite(xpos, ypos); 
     float freq = sI->getFreq(); 
     int amp = sI->getAmp(); 
     float pulse = sI->getPulse(); 

     switch(amp){ 
      case 0: 
       //printf("Hear sound in both ears.\n"); 
       data.targetBalance = .5; 
       break; 
      case 1: 
       //printf("Hear sound in left ear.\n"); 
       data.targetBalance = 0; 
       break; 
      case 2: 
       //printf("Hear sound in right ear.\n"); 
       data.targetBalance = 1; 
       break; 
      default: 
       //printf("Incorrect value for amp (left/right sound indicator)"); 
       data.targetBalance = .5; 
       break; 
     } 



     err = Pa_Initialize(); //scan for available devices i.e. audio jack, headphones 
     if(err != paNoError) { 
      printf("init\n"); 
      goto error; 
     } 
     //open the sound stream for processing 
     err = Pa_OpenDefaultStream(&stream, 0, 2, paFloat32, SAMPLE_RATE, 
      256, patestCallback, &data); //open the sound stream for processing 
     if(err != paNoError) { 
      printf("open\n"); 
      goto error; 
     } 

     //start the stream (i.e. play sound) if no errors 
     err = Pa_StartStream(stream); 
     if(err != paNoError) { 
      printf("start\n"); 
      goto error; 
     } 

     //check which ear(s) the sound should be played to 



     //hold that tone for a certain amount of time (pulse*200 millisec) 
     Pa_Sleep(pulse*200); 
     cout << "pulse: " << pulse << endl << "freq: " << freq << endl; 
     cout << "amp: " << amp << endl; 

     //stop the stream (i.e. stop playing sound) 
     err = Pa_StopStream(stream); 
     if(err != paNoError) { 
      printf("stop\n"); 
      goto error; 
     } 

     err = Pa_CloseStream(stream); 
     if(err != paNoError) { 
      printf("close\n"); 
      goto error; 
     } 

     err = Pa_Terminate(); 
     if(err != paNoError) { 
      printf("term\n"); 
      goto error; 
     } 
    } 
    int key = cvWaitKey(1); 
    // write 
    if (output_txt) 
     fprintf(output_txt, "%d %d %d %d\n", r.x, r.y, r.width, r.height); 
    if (output_avi) 
     cvWriteFrame(output_avi, drawImg); 

    // next 
    if (key == 'q'||key=='Q') 
     break; 
    frame = cvQueryFrame(capture); 
} 

Answer 1

似乎不一致的音頻播放是由於另一個代碼段沒有顯示在我的問題上面。下面是不正確的代碼。我相信這個錯誤與第一個if語句有關，並且在此函數中最後一個forloop。我認爲變量framesToCalc沒有被正確計算。因此，第一個for循環未將任何數據放入outputBuffer/out變量。然後，最後我清除剩餘的未使用的緩衝區空間。因此，由於緩衝區爲零，所以沒有聲音。我的解決方案是刪除第一個，如果還有最後一個forloop。另外，我做了第一個從i = 0到framesPerBuffer的循環。現在它完美地工作。

static int patestCallback(const void *inputBuffer, void *outputBuffer, unsigned long framesPerBuffer, const PaStreamCallbackTimeInfo *timeInfo, PaStreamCallbackFlags statusFlags, void *userData){ 
paTestData *data = (paTestData*)userData; 
SAMPLE_t *out = (SAMPLE_t *)outputBuffer; 
int i; 
int framesToCalc; 
int finished = 0; 
(void) inputBuffer; 
int left_phase = data->left_phase; 
int right_phase = data->right_phase; 


if(data->framesToGo < framesPerBuffer) 
{ 
    framesToCalc = data->framesToGo; 
    data->framesToGo = 0; 
    finished = 1; 
} 
else 
{ 
    framesToCalc = framesPerBuffer; 
    data->framesToGo -= framesPerBuffer; 
} 

for(i=0; i<framesToCalc; i++) 
{ 
    if(data->currentBalance < data->targetBalance) 
    { 
     data->currentBalance += BALANCE_DELTA; 
    } 
    else if(data->currentBalance > data->targetBalance) 
    { 
     data->currentBalance -= BALANCE_DELTA; 
    } 
    left_phase += (LEFT_FREQ/SAMPLE_RATE); 
    right_phase += (RIGHT_FREQ/SAMPLE_RATE); 
    if(fabs(data->currentBalance - .5) < .001){ 
     //left_phase += (double)(LEFT_FREQ/SAMPLE_RATE); 
     if(left_phase > 1.0) left_phase -= 1.0; 

     *out++ = DOUBLE_TO_SAMPLE(AMPLITUDE * sin((left_phase * M_PI * 2.))); 

     //right_phase += (double)(RIGHT_FREQ/SAMPLE_RATE); 
     if(right_phase > 1.0) right_phase -= 1.0; 
     *out++ = DOUBLE_TO_SAMPLE(AMPLITUDE * sin((right_phase * M_PI * 2.))); 
    }else{ 
     //left_phase += (double)(LEFT_FREQ/SAMPLE_RATE); 
     if(left_phase > 1.0) left_phase -= 1.0; 

     *out++ = DOUBLE_TO_SAMPLE(AMPLITUDE * sin((left_phase * M_PI * 2.))*(1.0 - data->currentBalance)); 

     //right_phase += (double)(RIGHT_FREQ/SAMPLE_RATE); 
     if(right_phase > 1.0) right_phase -= 1.0; 
     *out++ = DOUBLE_TO_SAMPLE(AMPLITUDE * sin((right_phase * M_PI * 2.))*data->currentBalance); 
    } 

} 
    // zero remainder of final buffer 
    for(; i<(int)framesPerBuffer; i++) 
    { 
     *out++ = SAMPLE_ZERO; //left 
     *out++ = SAMPLE_ZERO; //right 
    } 
    data->left_phase = left_phase; 
    data->right_phase = right_phase; 
    return finished; 
} 

Answer 2

爲什麼在幀循環中初始化和終止PortAudio？我會建議在程序開始時初始化一次，最後結束一次。