[Android] How to use MediaExtractor and MediaCodec ? - Audio part

In [Android] How to use MediaExtractor and MediaCodec? - Video part, we learned how to deal with the video data.
Let's check the audio part here.

Basically, we can use the same MediaExtractor to deal with video and audio.
We just need to switch the track dynamically using selectTrack().
However, it seems that we often missed the data somehow.
We will use separate MediaExtractor for video and audio individually.

The process for audio data is similar with that for video.
Audio data do not need to render to SurfaceView and we need an extra AudioTrack() for decoded PCM data.

private MediaExtractor extractorAudio;
private MediaCodec decoderAudio;

extractorAudio = new MediaExtractor();
extractorAudio.setDataSource("myTest.mp4");

for (int i = 0; i < extractorAudio.getTrackCount(); i++) {
 MediaFormat format = extractorAudio.getTrackFormat(i);
 String mime = format.getString(MediaFormat.KEY_MIME); 
 if (mime.startsWith("audio/")) {  
  audioTrack = i;  
  extractorAudio.selectTrack(audioTrack);
  formatAudio = format;        
  decoderAudio = MediaCodec.createDecoderByType(mime);
  sampleRate = format.getInteger(MediaFormat.KEY_SAMPLE_RATE);
  decoderAudio.configure(format, null, null, 0);
  break;
 }
}

if (audioTrack >=0) {
 if(decoderAudio == null)
 {
  Log.e(TAG, "Can't find audio info!");
  return;
 }
 else
 {
   // create our AudioTrack instance
   int minBufferSize = AudioTrack.getMinBufferSize(44100, AudioFormat.CHANNEL_CONFIGURATION_STEREO, AudioFormat.ENCODING_PCM_16BIT);
      int bufferSize = 4 * minBufferSize;
  playAudioTrack = new AudioTrack(
    AudioManager.STREAM_MUSIC,
    formatAudio.getInteger(MediaFormat.KEY_SAMPLE_RATE),
    AudioFormat.CHANNEL_OUT_STEREO,
    AudioFormat.ENCODING_PCM_16BIT,
    bufferSize,
    AudioTrack.MODE_STREAM
   );
  playAudioTrack.play();
  decoderAudio.start();
 }
}

Similarly, extractorAudio will find out the audio track according to MIME information.
We pass in the audio format using decoderAudio.configure(). No other parameters are necessary.

The data after decoderAudio are in PCM format.
We need AudioTrack() to play out the sound actually.

Below is the decode part:

ByteBuffer[] inputBuffersAudio=null;
ByteBuffer[] outputBuffersAudio=null;
BufferInfo infoAudio=null;


if (audioTrack >=0)
{
 inputBuffersAudio = decoderAudio.getInputBuffers();
 outputBuffersAudio = decoderAudio.getOutputBuffers();
 infoAudio = new BufferInfo();
}
boolean isEOS = false;
long startMs = System.currentTimeMillis();
long lasAudioStartMs = System.currentTimeMillis();
while (!Thread.interrupted()) { 
 if (audioTrack >=0)
 { 
  if (!isEOS) {
   int inIndex=-1;
   try {
    inIndex = decoderAudio.dequeueInputBuffer(10000);
   } catch (Exception e) {
    e.printStackTrace();    
   }
   if (inIndex >= 0) {    
    ByteBuffer buffer = inputBuffersAudio[inIndex];
    int sampleSize = extractorAudio.readSampleData(buffer, 0);
    if (sampleSize < 0) {
     
     decoderAudio.queueInputBuffer(inIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
     buffer.clear();
     isEOS = true;
    } else {     
     decoderAudio.queueInputBuffer(inIndex, 0, sampleSize, extractorAudio.getSampleTime(), 0);
     buffer.clear();
     extractorAudio.advance();
    }
    
   }
  }

  int outIndex=-1;
  try {
   outIndex = decoderAudio.dequeueOutputBuffer(infoAudio,10000);
  } catch (Exception e) {
   e.printStackTrace();   
  }

  switch (outIndex) {
  case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
   Log.d(TAG, "INFO_OUTPUT_BUFFERS_CHANGED");
   outputBuffersAudio = decoderAudio.getOutputBuffers();
   break;
  case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
   Log.d(TAG, "New format " + decoderAudio.getOutputFormat());
   playAudioTrack.setPlaybackRate(formatAudio.getInteger(MediaFormat.KEY_SAMPLE_RATE));
   break;
  case MediaCodec.INFO_TRY_AGAIN_LATER:
   Log.d(TAG, "dequeueOutputBuffer timed out!");
   break;
  default:
   if(outIndex>=0)
   {
    ByteBuffer buffer = outputBuffersAudio[outIndex];
    byte[] chunk = new byte[infoAudio.size];
    buffer.get(chunk);
    buffer.clear();
                if(chunk.length>0){         
                 playAudioTrack.write(chunk,0,chunk.length);
                }                
    decoderAudio.releaseOutputBuffer(outIndex, false);
   }
   break;
  }

  // All decoded frames have been rendered, we can stop playing now
  if ((infoAudio.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
   Log.d(TAG, "OutputBuffer BUFFER_FLAG_END_OF_STREAM");
   break;
  }
 }
}

if (audioTrack >=0)
{
 decoderAudio.stop();
 decoderAudio.release();
 playAudioTrack.stop();
}

extractorAudio.release();

The process are similar with video part. No more explanation here.
Need to notice is that for decoderAudio.releaseOutputBuffer(outIndex, false), the second parameter need to set to false. That is, no need to render.