[FFmpeg-devel] [PATCH v3 0/6] ffmpeg: late A/V encoder init, AVFrame metadata usage

[Jan Ekström]

On Fri, Oct 16, 2020, 16:16 Jan Ekström <jeebjp at gmail.com> wrote:

> This patch set started with a very simple wish to not have to set color
> related values manually each time when utilizing ffmpeg.c.
>
> As of the third iteration, the following changes were done since the
> second:
> 1. A simple mistake was corrected, fixing `debug_ts`.
> 2. As I noticed such a change enabling a fix for the interlaced flag
> writing
>    for Y4M, switched the location of the field order and
> interlaced/progressive
>    logic to where the encoder is initialized.
> 3. First attempt at fixing cases where the difference between stream copy
>    and re-encoding leads to the muxer queue filling up, breaking cases
>    where a stream with lots of small packets (such as audio) is copied,
>    and a seek ends up multiple seconds before the actual requested seek
>    time.
>
> Unfortunately, audio still needs two locations where the encoder is
> initialized, due to how avfilter_graph_request_oldest peeks and already
> puts
> one AVFrame to be available from the filter graph (which is then utilized
> as-is as an early return inside both av_buffersink_get_frame_flags and
> av_buffersink_get_samples). If this would be improved in lavfi (or the call
> to avfilter_graph_request_oldest removed), we could at least remove one of
> these.
>
> Currently limited to using values for video and started with the basic
> values,
> more can be added later if needed.
>
> This probably fixes some trac issues, but with a quick look I couldn't find
> anything that explicitly was due to lack of video color metadata
> passthrough.
>
>
> Jan
>
>
> Example 1:
>
> I have an RGB 3-D render, which I would like to encode into BT.709 YCbCr.
> The video filter I'm generally using for this (zscale) does flag the
> matrix in
> the output AVFrame.
>
> Yet to have the video encoder have the correct metadata set, I have to
> set the value(s) manually.
>
> With this patch set, the value(s) from the first AVFrame fed to
> do_video_out
> will be utilized.
>
> Example 2:
>
> I have an input video that sets one or more of the following:
> matrix/primaries/transfer function/range/chroma location.
>
> I just want to re-encode it. All of this metadata gets stripped.
>
> With this patch set, the value(s) from the first AVFrame fed to
> do_video_out
> will be utilized.
>
> Example 3:
>
> I have a video which has incorrect metadata tagged. Before, I had to set
> the correct data data manually.
>
> With this patch set, since ffmpeg.c takes color related options as
> dictionary
> keys, the AVFrame values will only be utilized if the user has not set the
> option for a given stream. Thus, this use case still works.
>
>
> Jan Ekström (6):
>   ffmpeg: deduplicate init_output_stream usage logic
>   ffmpeg: move AVFrame time base adjustment into a function
>   ffmpeg: move A/V non-streamcopy initialization to a later point
>   ffmpeg: pass decoded or filtered AVFrame to output stream
>     initialization
>   ffmpeg: move field order decision making to encoder initialization
>   ffmpeg: add a data size threshold for muxing queue size
>
>  doc/ffmpeg.texi                               |   5 +
>  fftools/ffmpeg.c                              | 249 ++++++++++++------
>  fftools/ffmpeg.h                              |  11 +
>  fftools/ffmpeg_opt.c                          |   8 +
>  .../fate/concat-demuxer-extended-lavf-mxf_d10 |   2 +-
>  .../fate/concat-demuxer-simple1-lavf-mxf_d10  |   2 +-
>  tests/ref/fate/rgb24-mkv                      |   4 +-
>  tests/ref/lavf/mxf_d10                        |   2 +-
>  tests/ref/lavf/mxf_dv25                       |   2 +-
>  tests/ref/lavf/mxf_dvcpro50                   |   2 +-
>  tests/ref/lavf/mxf_opatom                     |   2 +-
>  11 files changed, 202 insertions(+), 87 deletions(-)
>
> --
> 2.26.2
>

Ping on the patch set or if nothing else on the patch that improves the mux
queue buffer?

As far as I can see this can enable passing through information such as
stereo3d info as long as we get the info through AVFrame side data, but I
do not want to add even more changes to this patch set until it has
proceeded at least somewhat.

Jan

>