FFmpeg4入门系列教程21:音视频解混合(demuxer)为MP3和H264

索引地址:系列教程索引地址

上一篇:FFmpeg4入门系列教程20:视频添加滤镜

本文介绍将视频压缩数据(h264)和音频数据(mp3)从视频文件(mp4)中解出来。

FFmpeg4入门系列教程5:解码视频流过程中介绍了解码视频流的基本流程,但是只解码视频流。而在FFmpeg4入门系列教程15:音频解码为pcm中介绍了解码音频流的基本流程,并且是只解码音频流。而一个视频文件中包含音视频至少两条流,我们在解码的时候故意过滤了我们不需要的流。

随便找一个视频,查看视频信息:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
$ ffprobe Sample.mkv
ffprobe version n4.4 Copyright (c) 2007-2021 the FFmpeg developers
built with gcc 10.2.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
[flv @ 0x55894ec85440] Missing AMF_END_OF_OBJECT in AMF_DATA_TYPE_MIXEDARRAY, found 0
Input #0, flv, from 'Sample.mkv':
Metadata:
creator : www.qiyi.com
metadatacreator : Yet Another Metadata Injector for FLV - Version 1.2
hasKeyframes : true
hasVideo : true
hasAudio : true
hasMetadata : true
canSeekToEnd : false
datasize : 40600924
videosize : 36538352
audiosize : 3942540
lasttimestamp : 645
lastkeyframetimestamp: 637
lastkeyframelocation: 40109260
Duration: 00:10:44.87, start: 0.000000, bitrate: 503 kb/s
Stream #0:0: Video: h264 (High), yuv420p(tv, smpte170m/smpte170m/bt709, progressive), 624x480, 451 kb/s, 25 fps, 25 tbr, 1k tbn, 50 tbc
Stream #0:1: Audio: aac (HE-AAC), 44100 Hz, stereo, fltp, 47 kb/s

可以看到有两条流,分别是Stream #0:0 为h264即视频流,Stream #0:1为aac即音频流。接下来我们将其解出来并分别保存为音视频文件。

保存流程图为:

flow

保存函数调用图为:

functions

实际的操作为读取两条流,然后创建两条流,分别转换时间戳保存数据。

首先是打开输入

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
//打开输入流
ret = avformat_open_input(&ifmtCtx, inFilename, 0, 0);
if (ret < 0)
{
printf("can't open input file\n");
break;
}

//获取流信息
ret = avformat_find_stream_info(ifmtCtx, 0);
if (ret < 0)
{
printf("can't retrieve input stream information\n");
break;
}

创建输出上下文

1
2
3
4
5
6
avformat_alloc_output_context2(&ofmtCtxVideo, NULL, NULL, outFilenameVideo);
if (!ofmtCtxVideo)
{
printf("can't create video output context");
break;
}

打开输出上下文

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
int open_codec_context(int *streamIndex, AVFormatContext *&ofmtCtx, AVFormatContext *ifmtCtx, AVMediaType type)
{
AVStream *outStream = NULL, *inStream = NULL;
int ret = -1, index = -1;

index = av_find_best_stream(ifmtCtx, type, -1, -1, NULL, 0);
if (index < 0)
{
printf("can't find %s stream in input file\n", av_get_media_type_string(type));
return ret;
}

inStream = ifmtCtx->streams[index];

outStream = avformat_new_stream(ofmtCtx, NULL);
if (!outStream)
{
printf("failed to allocate output stream\n");
return ret;
}

ret = avcodec_parameters_copy(outStream->codecpar, inStream->codecpar);
if (ret < 0)
{
printf("failed to copy codec parametes\n");
return ret;
}

outStream->codecpar->codec_tag = 0;

// if (ofmtCtx->oformat->flags & AVFMT_GLOBALHEADER)
// {
// outStream->codec->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
// }

*streamIndex = index;

return 0;
}

打开输出文件

1
2
3
4
5
6
7
8
9
//打开输出文件:视频
if (!(ofmtCtxVideo->oformat->flags & AVFMT_NOFILE))
{
if (avio_open(&ofmtCtxVideo->pb, outFilenameVideo, AVIO_FLAG_WRITE) < 0)
{
printf("can't open output file: %s\n", outFilenameVideo);
break;
}
}

写文件头

1
2
3
4
5
6
//写文件头
if (avformat_write_header(ofmtCtxVideo, NULL) < 0)
{
printf("Error occurred when opening video output file\n");
break;
}

然后就是最关键的时间戳转换

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
while (1)
{
AVFormatContext *ofmtCtx;
AVStream *inStream, *outStream;

if (av_read_frame(ifmtCtx, &packet) < 0)
{
break;
}

inStream = ifmtCtx->streams[packet.stream_index];

if (packet.stream_index == videoIndex)
{
outStream = ofmtCtxVideo->streams[0];
ofmtCtx = ofmtCtxVideo;
}
else if (packet.stream_index == audioIndex)
{
outStream = ofmtCtxAudio->streams[0];
ofmtCtx = ofmtCtxAudio;
}
else
{
continue;
}

//convert PTS/DTS
packet.pts = av_rescale_q_rnd(packet.pts, inStream->time_base, outStream->time_base,(AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
packet.dts = av_rescale_q_rnd(packet.dts, inStream->time_base, outStream->time_base,(AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
packet.duration = av_rescale_q(packet.duration, inStream->time_base, outStream->time_base);
packet.pos = -1;
packet.stream_index = 0;

//write
if (av_interleaved_write_frame(ofmtCtx, &packet) < 0)
{
printf("Error muxing packet\n");
break;
}

av_packet_unref(&packet);
}

结果为:

result

可以用命令行播放看看效果。

aac

h264

完整代码在ffmpeg_Beginner中的21.video_demuxer_mp42h264mp3

下一篇:FFmpeg4入门系列教程22:音视频解混合(demuxer)为PCM和YUV420P