[FFmpeg-devel] [PATCH] ffmpeg: redesign video resampling in case of mid-stream video size/format change

[Stefano Sabatini]

On date Wednesday 2011-04-27 03:12:53 +0200, Michael Niedermayer encoded:
> On Wed, Apr 27, 2011 at 02:34:55AM +0200, Stefano Sabatini wrote:
[...]
> > Why I don't like av_vsrc_buffer_add_frame2():
> > 
> > * add a dependency on the scale filter
> > 
> > * the insertion of the filter in an already set-up filterchain is
> >   brittle (and doing several undocumented assumptions)
> > 
> > * it is an ad-hoc fix, indeed the same hack should be applied to other
> >   sources (e.g. movie, ffplay/input_filter and so on)
> > 
> > * other minor issues (not properly tested, never published for review,
> >   sloppy style and message reporting)
> 
> I dont disagree with any of these.
> yet, the scale filter is needed for libavfilter its a very central
> part needed for format convertion. And i think this is most important
> it is a fix that works today not a fix that works in months and for
> which we will in months know what issues it will have.
> 
> a better solution is very welcome but i think the current one should
> stay until a better one is actually in place and working.
> And ill try to fix bugs in  av_vsrc_buffer_add_frame2() until we have
> a better solution.

Another problem I didn't mentioned is that I would like to make
vsrc_buffer.h public, putting there a temporary solution is hindering
this objective (even if we could declare that the function is not
going to stay in the docs and the API is unstable).
 
> > The proper fix consists in dynamic mid-stream reconfiguration, which
> > should fix the problem also for ffplay (which now crashes/misbehaves
> > in that case). The nice thing with the video resampling context is
> > that it can be applied to ffplay as well, and fixes the problem at the
> > application level while a framework solution is developed (and fixes
> > some of the issues with the non-libavfilter path).
> 

> what is your oppinion of moving the format/size "equalization" into
> a seperate filter? (that could be placed after vsrc_buffer like any
> other filter, and would
> pass frames through unchanged when they match the output)

This would be acceptable (and could be applied also to "fix" the movie
souuce and the ffplay input).

> > So I propose to drop av_vsrc_buffer_add_frame2() in favor of this
> > solution, and fix properly the issue by extending the framework (which
> > I want to do, just after the unnecessary memcpy problem is fixed).
> > 
> > My plan for vsrc_buffer.c: remove dependency on AVFrame, document its
> > limitations (same input size/format assumption), and eventually
> > replace it in ffplay/ffmpeg by a more integrated vsrc_lavc_buffer
> > (which should allow direct rendering as implemented in ffplay.c).
> > 
> > vsrc_buffer would be still useful as a simple pure-libavfilter
> > solution (in case you don't want to depend on lavc).
> > 

> > > I dont understand how your solution that stores the resampling with
> > > input streams can work without the av_vsrc_buffer_add_frame2()
> > > when there are more output streams than input streams.
> > > The first output might want 100x100 the second 400x400, theres only
> > > one context so it can only scale to one.
> > 
> > Rescaling for the output streams is done in the libavfilter chain (a
> > scale filter is added to the end of the filterchain, one for each
> > output video stream), the rescale/reformat context is used only to
> > normalize the input video.
> 
> This means scaling videos twice if i understand it correctly

Not different from the current solution.
-- 
FFmpeg = Fascinating and Fantastic Magical Puristic Elitist Goblin