MediaCodec.configure失敗，IllegalStateException對於1080p視頻

Question

我一直在使用grafika的MoviePlayer和bigFlake的ExtractMpegFramesTest在我們的應用中實現尋找和提取幀功能。這些對我們大多數用戶來說都很好，但是其中一些在設置ExtractMpegFramesTest（發生在三星Galaxy S4 mini，三星Galaxy J7，三星Galaxy A5，華爲Ascend G7）上時遇到了IllegalStateException,MediaCodec.configure。相關代碼如下所示：

MoviePlayer

public void prepareResources() throws IOException { 
    // The MediaExtractor error messages aren't very useful. Check to see if the input 
    // file exists so we can throw a better one if it's not there. 
    if (!mSourceFile.canRead()) { 
     throw new FileNotFoundException("Unable to read " + mSourceFile); 
    } 

    try { 
     mExtractor = new MediaExtractor(); 
     mExtractor.setDataSource(mSourceFile.toString()); 
     mTrackIndex = selectTrack(mExtractor); 
     if (mTrackIndex < 0) { 
      throw new RuntimeException("No video track found in " + mSourceFile); 
     } 
     mExtractor.selectTrack(mTrackIndex); 

     MediaFormat format = mExtractor.getTrackFormat(mTrackIndex); 

     // Create a MediaCodec decoder, and configure it with the MediaFormat from the 
     // extractor. It's very important to use the format from the extractor because 
     // it contains a copy of the CSD-0/CSD-1 codec-specific data chunks. 
     String mime = format.getString(MediaFormat.KEY_MIME); 
     mDecoder = MediaCodec.createDecoderByType(mime); 
     mDecoder.configure(format, mOutputSurface, null, 0); 
     mDecoder.start(); 
     mDecoderInputBuffers = mDecoder.getInputBuffers(); 
    } catch (Exception e) { 
     e.printStackTrace(); 
    } 
} 

ExtractMpegFramesTest

private void extractMpegFrames() throws IOException { 
    MediaCodec decoder = null; 
    CodecOutputSurface outputSurface = null; 
    MediaExtractor extractor = null; 
    /*int saveWidth = 640; 
    int saveHeight = 480;*/ 

    try { 
     long start1 = System.currentTimeMillis(); 

     File inputFile = new File(mInputVideoPath); // must be an absolute path 
     // The MediaExtractor error messages aren't very useful. Check to see if the input 
     // file exists so we can throw a better one if it's not there. 
     if (!inputFile.canRead()) { 
      throw new FileNotFoundException("Unable to read " + inputFile); 
     } 

     extractor = new MediaExtractor(); 
     extractor.setDataSource(inputFile.toString()); 
     int trackIndex = selectTrack(extractor); 
     if (trackIndex < 0) { 
      throw new RuntimeException("No video track found in " + inputFile); 
     } 
     extractor.selectTrack(trackIndex); 

     MediaFormat format = extractor.getTrackFormat(trackIndex); 
     if (VERBOSE) { 
      Log.d(TAG, "Video size is " + format.getInteger(MediaFormat.KEY_WIDTH) + "x" + 
        format.getInteger(MediaFormat.KEY_HEIGHT)); 
     } 

     // Could use width/height from the MediaFormat to get full-size frames. 
     outputSurface = new CodecOutputSurface(format.getInteger(MediaFormat.KEY_WIDTH), format.getInteger(MediaFormat.KEY_HEIGHT)); 

     // Create a MediaCodec decoder, and configure it with the MediaFormat from the 
     // extractor. It's very important to use the format from the extractor because 
     // it contains a copy of the CSD-0/CSD-1 codec-specific data chunks. 
     String mime = format.getString(MediaFormat.KEY_MIME); 
     decoder = MediaCodec.createDecoderByType(mime); 
     decoder.configure(format, outputSurface.getSurface(), null, 0); //fails right here 
     decoder.start(); 

     long end1 = System.currentTimeMillis(); 
     Timber.d("extractMpegFrames(): FRAME_EXTRACT setup in: %d millis", (end1-start1)); 

     long start2 = System.currentTimeMillis(); 
     doExtract(extractor, trackIndex, decoder, outputSurface); 
     long end2 = System.currentTimeMillis(); 
     Timber.d("extractMpegFrames(): FRAME_EXTRACT doExtract in: %d millis", (end2-start2)); 
    } finally { 
     // release everything we grabbed 
     if (outputSurface != null) { 
      outputSurface.release(); 
      outputSurface = null; 
     } 
     if (decoder != null) { 
      decoder.stop(); 
      decoder.release(); 
      decoder = null; 
     } 
     if (extractor != null) { 
      extractor.release(); 
      extractor = null; 
     } 
    } 
} 

我知道這個question的，但我不明白的是爲什麼MediaCodec可以配置OK在MoviePlayer（視頻運行正常），但在ExtractMpegFramesTest失敗。起初，我認爲該設備在設置OpenGL時遇到了一些問題，但最終成爲MediaCodec問題。

任何有識之士將不勝感激。

編輯：獲得銀河S4迷你測試設備後，我已經能夠獲得更多有用的日誌：

D/MoviePlayer: Extractor selected track 0 (video/avc): {max-input-size=1572864, height=1080, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20], width=1920, durationUs=5131211, csd-1=java.nio.ByteArrayBuffer[position=0,limit=9,capacity=9], mime=video/avc, isDMCMMExtractor=1} 
D/MoviePlayer: Video size is 1920x1080 
D/MoviePlayer: Extractor selected track 0 (video/avc): {max-input-size=1572864, height=1080, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20], width=1920, durationUs=5131211, csd-1=java.nio.ByteArrayBuffer[position=0,limit=9,capacity=9], mime=video/avc, isDMCMMExtractor=1} 
I/OMXClient: Using client-side OMX mux. 
E/ACodec: [OMX.qcom.video.decoder.avc] storeMetaDataInBuffers failed w/ err -2147483648 
E/ACodec: configureCodec multi window instance fail appPid : 3283 
E/ACodec: [OMX.qcom.video.decoder.avc] configureCodec returning error -38 
E/MediaCodec: Codec reported an error. (omx error 0x80001001, internalError -38) 
W/System.err: java.lang.IllegalStateException 
W/System.err:  at android.media.MediaCodec.native_configure(Native Method) 
W/System.err:  at android.media.MediaCodec.configure(MediaCodec.java:262) 
W/System.err:  at co.test.testing.player.MoviePlayer.prepareResources(MoviePlayer.java:237) 
W/System.err:  at co.test.testing.activities.VideoActivity.surfaceCreated(VideoActivity.java:276) 
W/System.err:  at android.view.SurfaceView.updateWindow(SurfaceView.java:602) 
W/System.err:  at android.view.SurfaceView.access$000(SurfaceView.java:94) 
W/System.err:  at android.view.SurfaceView$3.onPreDraw(SurfaceView.java:183) 
W/System.err:  at android.view.ViewTreeObserver.dispatchOnPreDraw(ViewTreeObserver.java:891) 
W/System.err:  at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:2201) 
W/System.err:  at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:1256) 
W/System.err:  at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:6635) 
W/System.err:  at android.view.Choreographer$CallbackRecord.run(Choreographer.java:813) 
W/System.err:  at android.view.Choreographer.doCallbacks(Choreographer.java:613) 
W/System.err:  at android.view.Choreographer.doFrame(Choreographer.java:583) 
W/System.err:  at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:799) 
W/System.err:  at android.os.Handler.handleCallback(Handler.java:733) 
W/System.err:  at android.os.Handler.dispatchMessage(Handler.java:95) 
W/System.err:  at android.os.Looper.loop(Looper.java:146) 
W/System.err:  at android.app.ActivityThread.main(ActivityThread.java:5593) 
W/System.err:  at java.lang.reflect.Method.invokeNative(Native Method) 
W/System.err:  at java.lang.reflect.Method.invoke(Method.java:515) 
W/System.err:  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1283) 
W/System.err:  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1099) 
W/System.err:  at dalvik.system.NativeStart.main(Native Method) 

這些是有這個問題的設備檢索到的編解碼器信息：

• OMX.qcom.video.encoder.avc

• OMX.SEC.avc.enc

• OMX.Exynos.AVC.Encoder

奇怪的是銀河S4迷你可處理720p的視頻就好了，它只是具有1080p的視頻問題。

Answer 1

好吧，這裏再回答我的問題：

• 首先，我爲什麼用MoviePlayer顯示視頻後也無法提取與ExtractMpegFramesTest框架：

  看來，一些設備可以」 t在處理高分辨率視頻（在Galaxy S4 mini的情況下> 720p）同時處理MediaCodec的兩個實例，因此您必須在啓動另一個實例之前正確釋放一個MediaCodec實例，如下所示：

  public void releaseResources() { 
      // release everything we grabbed 
      if (mDecoder != null) { 
       try { 
        mDecoder.stop(); 
        mDecoder.release(); 
        mDecoder = null; 
       } catch (Exception e) { 
        Timber.d("releaseResources(): message %s cause %s", e.getMessage(), e.getCause()); 
        e.printStackTrace(); 
       } 
      } 
      if (mExtractor != null) { 
       try { 
        mExtractor.release(); 
        mExtractor = null; 
       } catch (Exception e) { 
        Timber.d("releaseResources(): message %s cause %s", e.getMessage(), e.getCause()); 
        e.printStackTrace(); 
       } 
      } 
  } 
  

• 其次，爲什麼MoviePlayer未能配置1080p視頻。在我的情況下，因爲我從一個VideoView的媒體選擇器活動開始此活動，其中MediaPlayer.OnErrorListener()已被註冊以檢測錯誤的視頻。問題是onError()回調在Galaxy S4 mini上有一些非常奇怪的行爲，但它在我的開發設備上工作得非常好。我猜想這個MediaPlayer.OnErrorListener()離開MediaCodec處於一個糟糕的狀態，以後會引起很多頭痛。

  所以解決方案是設置一個空白OnErrorListener()並在做其他任何事情之前正確地釋放VideoViewMediaCodec。事情是這樣的：

  mVideoView.setOnErrorListener(new MediaPlayer.OnErrorListener() { 
       @Override 
       public boolean onError(MediaPlayer mp, int what, int extra) { 
        return true; 
       } 
  }); 
  ..... 
  mVideoView.stopPlayback(); 
  doOtherThingWithMediaCodec(); 
  

這說明很多不同的設備上所發生奇怪的事情。我希望這可以幫助別人調試他們的應用程序。