[FFmpeg-devel] [EXT] Re: [PATCH v3 1/3] avcodec/v4l2_context: don't reinit output queue when dynamic resolution change

[Andriy Gelman]

On Mon, 02. Aug 02:32, Ming Qian wrote:
> > -----Original Message-----
> > From: Andriy Gelman [mailto:andriy.gelman at gmail.com]
> > Sent: Monday, August 2, 2021 7:36 AM
> > To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
> > Cc: Ming Qian <ming.qian at nxp.com>
> > Subject: [EXT] Re: [FFmpeg-devel] [PATCH v3 1/3] avcodec/v4l2_context: don't
> > reinit output queue when dynamic resolution change
> > 
> > Caution: EXT Email
> > 
> > Hi,
> > 
> > On Thu, 29. Jul 14:00, Ming Qian wrote:
> > > in the v4l2 stateful video document, we can see the following
> > > description:
> > >     During the resolution change sequence, the OUTPUT queue must
> > remain
> > >     streaming. Calling VIDIOC_STREAMOFF() on the OUTPUT queue would
> > >     abort the sequence and initiate a seek.
> > >
> > >     In principle, the OUTPUT queue operates separately from the CAPTURE
> > >     queue and this remains true for the duration of the entire
> > >     resolution change sequence as well.
> > >
> > > so don't reinit the output queue when handling the resolution change
> > > event
> > >
> > > Signed-off-by: Ming Qian <ming.qian at nxp.com>
> > > ---
> > >  libavcodec/v4l2_context.c | 27 ++-------------------------
> > >  1 file changed, 2 insertions(+), 25 deletions(-)
> > >
> > > diff --git a/libavcodec/v4l2_context.c b/libavcodec/v4l2_context.c
> > > index ff1ea8e57b08..dda5157698c3 100644
> > > --- a/libavcodec/v4l2_context.c
> > > +++ b/libavcodec/v4l2_context.c
> > > @@ -162,9 +162,8 @@ static int v4l2_handle_event(V4L2Context *ctx)  {
> > >      V4L2m2mContext *s = ctx_to_m2mctx(ctx);
> > >      struct v4l2_format cap_fmt = s->capture.format;
> > > -    struct v4l2_format out_fmt = s->output.format;
> > >      struct v4l2_event evt = { 0 };
> > > -    int full_reinit, reinit, ret;
> > > +    int reinit, ret;
> > >
> > >      ret = ioctl(s->fd, VIDIOC_DQEVENT, &evt);
> > >      if (ret < 0) {
> > > @@ -180,25 +179,12 @@ static int v4l2_handle_event(V4L2Context *ctx)
> > >      if (evt.type != V4L2_EVENT_SOURCE_CHANGE)
> > >          return 0;
> > >
> > > -    ret = ioctl(s->fd, VIDIOC_G_FMT, &out_fmt);
> > > -    if (ret) {
> > > -        av_log(logger(ctx), AV_LOG_ERROR, "%s VIDIOC_G_FMT\n",
> > s->output.name);
> > > -        return 0;
> > > -    }
> > > -
> > >      ret = ioctl(s->fd, VIDIOC_G_FMT, &cap_fmt);
> > >      if (ret) {
> > >          av_log(logger(ctx), AV_LOG_ERROR, "%s VIDIOC_G_FMT\n",
> > s->capture.name);
> > >          return 0;
> > >      }
> > >
> > > -    full_reinit = v4l2_resolution_changed(&s->output, &out_fmt);
> > > -    if (full_reinit) {
> > > -        s->output.height = v4l2_get_height(&out_fmt);
> > > -        s->output.width = v4l2_get_width(&out_fmt);
> > > -        s->output.sample_aspect_ratio = v4l2_get_sar(&s->output);
> > > -    }
> > > -
> > >      reinit = v4l2_resolution_changed(&s->capture, &cap_fmt);
> > >      if (reinit) {
> > >          s->capture.height = v4l2_get_height(&cap_fmt);
> > > @@ -206,18 +192,9 @@ static int v4l2_handle_event(V4L2Context *ctx)
> > >          s->capture.sample_aspect_ratio = v4l2_get_sar(&s->capture);
> > >      }
> > >
> > > -    if (full_reinit || reinit)
> > > +    if (reinit)
> > >          s->reinit = 1;
> > >
> > > -    if (full_reinit) {
> > > -        ret = ff_v4l2_m2m_codec_full_reinit(s);
> > > -        if (ret) {
> > > -            av_log(logger(ctx), AV_LOG_ERROR,
> > "v4l2_m2m_codec_full_reinit\n");
> > > -            return AVERROR(EINVAL);
> > > -        }
> > > -        goto reinit_run;
> > > -    }
> > > -
> > >      if (reinit) {
> > >          if (s->avctx)
> > >              ret = ff_set_dimensions(s->avctx, s->capture.width,
> > s->capture.height);
> > > --
> > > 2.32.0
> > >
> > 
> > You probably missed my question in the v2 thread. I asked:
> > 
> > Could you send me a sample for testing, please.
> > What boards did you test on?
> > 
> > Thanks,
> > --
> > Andriy
> 
> Hi Andriy,
>    I tested on IMX8QXP and IMX8MP, and they are based on different codec IP.
>    On imx8qxp, the vpu doesn't handle the buffer of v4l2 output queue, the driver need to copy the stream data to a ring buffer.
>    But on imx8mp, the vpu can decode the buffer of v4l2 output queue directly.
> 
> My test script is as below:
> 
>  ffmpeg -vsync passthrough -vcodec h264_v4l2m2m -i /mnt/test_352_288.h264  -f rawvideo -y output_nv12.yuv
> 
> the test_352_288.h264 is attached, but it doesn't really matter which stream file is tested.

Sorry for the delay replying.

I tested on RPi4 with an example where there is a resolution change
midstream, and for me this doesn't work at all (with or without your patch).
The code gets stuck trying to wait for the buffers to get released before reinit is done:

3833e70] capture changed (1280x720) -> (640x368)
[h264_v4l2m2m @ 0x3833e70] reinit context
[h264_v4l2m2m @ 0x3833e70] waiting for user to release AVBufferRefs

I don't know how much the resolution change part was tested in the past.

I'll share the sample with you by email.
I tried to upload the sample to https://streams.videolan.org/upload/, but it
doesn't say what's the url of the file after uploading. It doesn't exist in ffmpeg directory..
Anyone used this before?

Thanks,
-- 
Andriy