[FFmpeg-devel] [PATCH] lavc/libvpxenc: prevent fifo from filling up

[James Zern]

On Thu, Jul 6, 2023 at 12:45 AM Anton Khirnov <anton at khirnov.net> wrote:
>
> Quoting James Zern (2023-07-05 21:16:37)
> > On Wed, Jul 5, 2023 at 12:15 PM James Zern <jzern at google.com> wrote:
> > >
> > > Hi,
> > >
> >
> > +ffmpeg-dev. I took the wrong email off the reply.
> >
> > > On Mon, Jul 3, 2023 at 10:08 PM David Lemler <david at lemler.family> wrote:
> > > >
> > > > Prevent the fifo used in encoding VPx videos from filling up and stopping
> > > > encode when it reaches 21845 items, which happens when the video has more
> > > > than
> > > > that number of frames.
> > > >
> > >
> > > What failure do you see, a memory allocation error? 21845 sounds
> > > somewhat arbitrary, maybe specific to your machine, so I'd leave it
> > > off the comment and commit message.
> > >
> > > > This problem occurs when performing the first pass of a 2-pass encode, as
> > > > otherwise, the fifo is properly drained, preventing it from overflowing.
> > > >
> > > > Problem is fixed by manually draining the fifo when performing the first
> > > > pass
> > > > of a 2-pass encode.
> > > >
> > > > Fixes the regression originally introduced in
> > > > 5bda4ec6c3cb6f286bb40dee4457c3c26e0f78cb
> > > >
> > >
> > > Anton, any comments here?
>
> Is it guaranteed by the libvpx API that no encoded packets are returned
> for the first pass of 2pass encoding? If so, then it probably makes most
> sense not to send anything to the fifo in the first place.
>

You'll only get stats packets in the first pass, no frames.
David, you can check `!(avctx->flags & AV_CODEC_FLAG_PASS1)` before
calling `frame_data_submit()`.