作者：leisure 本文为作者原创，转载请注明出处：https://www.cnblogs.com/leisure_chn/p/10318145.html 所谓内存IO，在FFmpeg中叫作“buffered IO”或“custom IO”，指的是将一块内存缓冲区用作FFmpeg的输入或输出。与内存IO操作对应的是指定URL作为FFmpeg的输入或输出，比如URL可能是普通文件或网络流地址等。这两种输入输出模式我们暂且称作“内存IO模式”和“URL-IO模式”。 本文源码基于FFmpeg 4.1版本，为帮助理解，可参考FFmpeg工程examples中如下两份代码： https://github.com/FFmpeg/FFmpeg/blob/n4.1/doc/examples/avio_reading.c https://github.com/FFmpeg/FFmpeg/blob/n4.1/doc/examples/remuxing.c 1. 内存区作输入 1.1 用法 用法如示例中注释的步骤，如下： 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 // @opaque : 是由用户提供的参数，指向用户数据 // @buf : 作为FFmpeg的输入，此处由用户准备好buf中的数据 // @buf_size: buf的大小 // @return : 本次IO数据量 static int read_packet(void *opaque, uint8_t *buf, int buf_size) { int fd = *((int *)opaque); int ret = read(fd, buf, buf_size); return ret; } int main() { AVFormatContext *ifmt_ctx = NULL; AVIOContext *avio_in = NULL; uint8_t *ibuf = NULL; size_t ibuf_size = 4096; int fd = -1; // 打开一个FIFO文件的读端 fd = open_fifo_for_read("/tmp/test_fifo"); // 1. 分配缓冲区 ibuf = av_malloc(ibuf_size); // 2. 分配AVIOContext，第三个参数write_flag为0 avio_in = avio_alloc_context(ibuf, ibuf_size, 0, &fd, &read_packet, NULL, NULL); // 3. 分配AVFormatContext，并指定AVFormatContext.pb字段。必须在调用avformat_open_input()之前完成 ifmt_ctx = avformat_alloc_context(); ifmt_ctx->pb = avio_in; // 4. 打开输入(读取封装格式文件头) avformat_open_input(&ifmt_ctx, NULL, NULL, NULL); ...... } 当启用内存IO模式后(即ifmt_ctx->pb有效时)，将会忽略avformat_open_input()第二个参数url的值。在上述示例中，打开了FIFO的读端，并在回调函数中将FIFO中的数据填入内存缓冲区ibuf，内存缓冲区ibuf将作为FFmpeg的输入。在上述示例中，因为打开的是一个命名管道FIFO，FIFO的数据虽然在内存中，但FIFO有名字("/tmp/test_fifo")，所以此例也可以使用URL-IO模式，如下： 1 2 AVFormatContext *ifmt_ctx = NULL; avformat_open_input(&ifmt_ctx, "/tmp/test_fifo", NULL, NULL); 而对于其他一些场合，当有效音视频数据位于内存，而这片内存并无一个URL属性可用时，则只能使用内存IO模式来取得输入数据。 1.2 回调时机 回调函数何时被回调呢？所有需要从输入源中读取数据的时刻，都将调用回调函数。和输入源是普通文件相比，只不过输入源变成了内存区，其他各种外在表现并无不同。 如下各函数在不同的阶段从输入源读数据，都会调用回调函数： avformat_open_input() 从输入源读取封装格式文件头 avformat_find_stream_info() 从输入源读取一段数据，尝试解码，以获取流信息 av_read_frame() 从输入源读取数据包 2. 内存区作输出 2.1 用法 用法如示例中注释的步骤，如下： 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 // @opaque : 是由用户提供的参数，指向用户数据 // @buf : 作为FFmpeg的输出，此处FFmpeg已准备好buf中的数据 // @buf_size: buf的大小 // @return : 本次IO数据量 static int write_packet(void *opaque, uint8_t *buf, int buf_size) { int fd = *((int *)opaque); int ret = write(fd, buf, buf_size); return ret; } int main() { AVFormatContext *ofmt_ctx = NULL; AVIOContext *avio_out = NULL; uint8_t *obuf = NULL; size_t obuf_size = 4096; int fd = -1; // 打开一个FIFO文件的写端 fd = open_fifo_for_write("/tmp/test_fifo"); // 1. 分配缓冲区 obuf = av_malloc(obuf_size); // 2. 分配AVIOContext，第三个参数write_flag为1 AVIOContext *avio_out = avio_alloc_context(obuf, obuf_size, 1, &fd, NULL, write_packet, NULL); // 3. 分配AVFormatContext，并指定AVFormatContext.pb字段。必须在调用avformat_write_header()之前完成 avformat_alloc_output_context2(&ofmt_ctx, NULL, NULL, NULL); ofmt_ctx->pb=avio_out; // 4. 将文件头写入输出文件 avformat_write_header(ofmt_ctx, NULL); ...... } 当启用内存IO模式后(即ofmt_ctx->pb有效时)，FFmpeg会将输出写入内存缓冲区obuf，用户可在回调函数中将obuf中的数据取走。在上述示例中，因为打开的是一个命名管道FIFO，FIFO的数据虽然在内存中，但FIFO有名字("/tmp/test_fifo")，所以此例也可以使用URL-IO模式，如下： 1 2 AVFormatContext *ofmt_ctx = NULL; avformat_alloc_output_context2(&ofmt_ctx, "/tmp/test_fifo", NULL, NULL); 而对于其他一些场合，需将数据输出到内存，而这片内存并无一个URL属性可用时，则只能使用内存IO模式。 2.2 回调时机 回调函数何时被回调呢？所有输出数据的时刻，都将调用回调函数。和输出是普通文件相比，只不过输出变成了内存区，其他各种外在表现并无不同。 如下各函数在不同的阶段会输出数据，都会调用回调函数： avformat_write_header() 将流头部信息写入输出区 av_interleaved_write_frame() 将数据包写入输出区 av_write_trailer() 将流尾部信息写入输出区 3. 实现机制 如下是与内存IO操作相关的一些关键数据结构及函数，我们从API接口层面来看一下内存IO的实现机制，而不深入分析内部源码。FFmpeg的API注释非常详细，从注释中能得到很多有用信息。 3.1 struct AVIOContext 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 /** * Bytestream IO Context. * New fields can be added to the end with minor version bumps. * Removal, reordering and changes to existing fields require a major * version bump. * sizeof(AVIOContext) must not be used outside libav*. * * @note None of the function pointers in AVIOContext should be called * directly, they should only be set by the client application * when implementing custom I/O. Normally these are set to the * function pointers specified in avio_alloc_context() */ typedef struct AVIOContext { ...... /* * The following shows the relationship between buffer, buf_ptr, * buf_ptr_max, buf_end, buf_size, and pos, when reading and when writing * (since AVIOContext is used for both): * ********************************************************************************** * READING ********************************************************************************** * * | buffer_size | * |---------------------------------------| * | | * * buffer buf_ptr buf_end * +---------------+-----------------------+ * |/ / / / / / / /|/ / / / / / /| | * read buffer: |/ / consumed / | to be read /| | * |/ / / / / / / /|/ / / / / / /| | * +---------------+-----------------------+ * * pos * +-------------------------------------------+-----------------+ * input file: | | | * +-------------------------------------------+-----------------+ * * ********************************************************************************** * WRITING ********************************************************************************** * * | buffer_size | * |--------------------------------------| * | | * * buf_ptr_max * buffer (buf_ptr) buf_end * +-----------------------+--------------+ * |/ / / / / / / / / / / /| | * write buffer: | / / to be flushed / / | | * |/ / / / / / / / / / / /| | * +-----------------------+--------------+ * buf_ptr can be in this * due to a backward seek * * pos * +-------------+----------------------------------------------+ * output file: | | | * +-------------+----------------------------------------------+ * */ unsigned char *buffer; /**< Start of the buffer. */ int buffer_size; /**< Maximum buffer size */ unsigned char *buf_ptr; /**< Current position in the buffer */ unsigned char *buf_end; /**< End of the data, may be less than buffer+buffer_size if the read function returned less data than requested, e.g. for streams where no more data has been received yet. */ void *opaque; /**< A private pointer, passed to the read/write/seek/... functions. */ int (*read_packet)(void *opaque, uint8_t *buf, int buf_size); int (*write_packet)(void *opaque, uint8_t *buf, int buf_size); ...... } AVIOContext; 注意：此数据结构中的成员不应由用户程序直接访问。当使用内存IO模式时，用户应调用avio_alloc_context()对此结构的read_packet和write_packet函数指针进行赋值。 3.2 AVIOContext* AVFormatContext.pb 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 /** * Format I/O context. * ...... */ typedef struct AVFormatContext { ...... /** * I/O context. * * - demuxing: either set by the user before avformat_open_input() (then * the user must close it manually) or set by avformat_open_input(). * - muxing: set by the user before avformat_write_header(). The caller must * take care of closing / freeing the IO context. * * Do NOT set this field if AVFMT_NOFILE flag is set in * iformat/oformat.flags. In such a case, the (de)muxer will handle * I/O in some other way and this field will be NULL. */ AVIOContext *pb; ...... } struct AVFormatContext结构中与内存IO操作相关的重要成员是AVIOContext *pb，有如下规则： 解复用过程：在调用avformat_open_input()前由用户手工设置，因为从avformat_open_input()开始有读输入的操作。 复用过程：在调用avformat_write_header()前由用户手工设置，因为从avformat_write_header()开始有写输出的操作。 3.3 输入时：avformat_open_input() 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 /** * Open an input stream and read the header. The codecs are not opened. * The stream must be closed with avformat_close_input(). * * @param ps Pointer to user-supplied AVFormatContext (allocated by avformat_alloc_context). * May be a pointer to NULL, in which case an AVFormatContext is allocated by this * function and written into ps. * Note that a user-supplied AVFormatContext will be freed on failure. * @param url URL of the stream to open. * @param fmt If non-NULL, this parameter forces a specific input format. * Otherwise the format is autodetected. * @param options A dictionary filled with AVFormatContext and demuxer-private options. * On return this parameter will be destroyed and replaced with a dict containing * options that were not found. May be NULL. * * @return 0 on success, a negative AVERROR on failure. * * @note If you want to use custom IO, preallocate the format context and set its pb field. */ int avformat_open_input(AVFormatContext **ps, const char *url, AVInputFormat *fmt, AVDictionary **options); 打开输入流读取头部信息。如果使用内存IO模式，应在此之前分配AVFormatContext并设置其pb成员。 3.4 输出时：avformat_write_header() 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 /** * Allocate the stream private data and write the stream header to * an output media file. * * @param s Media file handle, must be allocated with avformat_alloc_context(). * Its oformat field must be set to the desired output format; * Its pb field must be set to an already opened AVIOContext. * @param options An AVDictionary filled with AVFormatContext and muxer-private options. * On return this parameter will be destroyed and replaced with a dict containing * options that were not found. May be NULL. * * @return AVSTREAM_INIT_IN_WRITE_HEADER on success if the codec had not already been fully initialized in avformat_init, * AVSTREAM_INIT_IN_INIT_OUTPUT on success if the codec had already been fully initialized in avformat_init, * negative AVERROR on failure. * * @see av_opt_find, av_dict_set, avio_open, av_oformat_next, avformat_init_output. */ av_warn_unused_result int avformat_write_header(AVFormatContext *s, AVDictionary **options); 将流头部信息写入输出文件。在调用此函数前，AVFormatContext.pb成员必须设置为一个已经打开的AVIOContext。AVFormatContext.pb赋值方式分为两种情况： [1]. URL-IO模式：调用avio_open()或avio_open2()，形如 avio_open(&ofmt_ctx->pb, out_filename, AVIO_FLAG_WRITE); [2]. 内存IO模式：调用avio_alloc_context()分配AVIOContext，然后为pb赋值，形如： 1 2 avformat_alloc_output_context2(&ofmt_ctx, NULL, NULL, NULL); ofmt_ctx->pb=avio_out; 3.5 内存IO模式：avio_alloc_context() 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 /** * Allocate and initialize an AVIOContext for buffered I/O. It must be later * freed with avio_context_free(). * * @param buffer Memory block for input/output operations via AVIOContext. * The buffer must be allocated with av_malloc() and friends. * It may be freed and replaced with a new buffer by libavformat. * AVIOContext.buffer holds the buffer currently in use, * which must be later freed with av_free(). * @param buffer_size The buffer size is very important for performance. * For protocols with fixed blocksize it should be set to this blocksize. * For others a typical size is a cache page, e.g. 4kb. * @param write_flag Set to 1 if the buffer should be writable, 0 otherwise. * @param opaque An opaque pointer to user-specific data. * @param read_packet A function for refilling the buffer, may be NULL. * For stream protocols, must never return 0 but rather * a proper AVERROR code. * @param write_packet A function for writing the buffer contents, may be NULL. * The function may not change the input buf