[FFmpeg-devel] [PATCH] ffmpeg: redesign video resampling in case of mid-stream video size/format change

[Michael Niedermayer]

On Wed, Apr 27, 2011 at 02:34:55AM +0200, Stefano Sabatini wrote:
> On date Tuesday 2011-04-26 18:42:07 +0200, Michael Niedermayer encoded:
> > On Tue, Apr 26, 2011 at 12:17:39AM +0200, Stefano Sabatini wrote:
> > > On date Monday 2011-04-25 00:26:05 +0200, Stefano Sabatini encoded:
> > > [...]
> > > > New approach, this is the neatest design I can find without
> > > > introducing filterchain re-configuration. It also fixes some problems
> > > > in the non-libavfilter code path.
> > > > -- 
> > > > FFmpeg = Formidable Fancy Majestic Prodigious Eager Game
> > > 
> > > > From b3f4b7091122ca341b3a10b4f6edbbcb7183af2f Mon Sep 17 00:00:00 2001
> > > > From: Stefano Sabatini <stefano.sabatini-lala at poste.it>
> > > > Date: Sat, 16 Apr 2011 23:14:44 +0200
> > > > Subject: [PATCH] ffmpeg: redesign video resampling in case of mid-stream video size/format change
> > > > 
> > > > The new design implements a new video rescaling API and supports video
> > > > resolution/format change with and without AVFILTER_CONFIG, thus
> > > > avoiding the need for the av_vsrc_buffer_add_frame2() hack.
> > > > 
> > > > If libavfilter is not enabled, resampling is performed in
> > > > do_video_out(), before to send the decoded frame to the output. In
> > > > this case the resampling API will also perform the scaling to the
> > > > output video size/format.
> > > > 
> > > > If libavfilter is enabled, resampling is performed in
> > > > pre_process_video_frame(), before the video is sent to the
> > > > filterchain. This way we ensure that the filterchain is feeded with
> > > > frames of the same size/format, thus avoiding the need of dynamic
> > > > filterchain reconfiguration.
> > > > 
> > > > Fix issues:
> > > > roundup #2217
> > > > trac #59
> > > > trac #64
> > > > trac #74
> > > 
> > > Ping.
> > 
> > ive fixed several small bugs in av_vsrc_buffer_add_frame2() and it
> > seems that fixed these issues.
> > It should be easy to add interlaced scaling support and the code
> > can reuse an existing scale filter instead of needing an additional
> > filter.
> 
> Why I don't like av_vsrc_buffer_add_frame2():
> 
> * add a dependency on the scale filter
> 
> * the insertion of the filter in an already set-up filterchain is
>   brittle (and doing several undocumented assumptions)
> 
> * it is an ad-hoc fix, indeed the same hack should be applied to other
>   sources (e.g. movie, ffplay/input_filter and so on)
> 
> * other minor issues (not properly tested, never published for review,
>   sloppy style and message reporting)

I dont disagree with any of these.
yet, the scale filter is needed for libavfilter its a very central
part needed for format convertion. And i think this is most important
it is a fix that works today not a fix that works in months and for
which we will in months know what issues it will have.

a better solution is very welcome but i think the current one should
stay until a better one is actually in place and working.
And ill try to fix bugs in  av_vsrc_buffer_add_frame2() until we have
a better solution.


> 
> The proper fix consists in dynamic mid-stream reconfiguration, which
> should fix the problem also for ffplay (which now crashes/misbehaves
> in that case). The nice thing with the video resampling context is
> that it can be applied to ffplay as well, and fixes the problem at the
> application level while a framework solution is developed (and fixes
> some of the issues with the non-libavfilter path).

what is your oppinion of moving the format/size "equalization" into
a seperate filter? (that could be placed after vsrc_buffer like any
other filter, and would
pass frames through unchanged when they match the output)



> 
> So I propose to drop av_vsrc_buffer_add_frame2() in favor of this
> solution, and fix properly the issue by extending the framework (which
> I want to do, just after the unnecessary memcpy problem is fixed).
> 
> My plan for vsrc_buffer.c: remove dependency on AVFrame, document its
> limitations (same input size/format assumption), and eventually
> replace it in ffplay/ffmpeg by a more integrated vsrc_lavc_buffer
> (which should allow direct rendering as implemented in ffplay.c).
> 
> vsrc_buffer would be still useful as a simple pure-libavfilter
> solution (in case you don't want to depend on lavc).
> 
> > I dont understand how your solution that stores the resampling with
> > input streams can work without the av_vsrc_buffer_add_frame2()
> > when there are more output streams than input streams.
> > The first output might want 100x100 the second 400x400, theres only
> > one context so it can only scale to one.
> 
> Rescaling for the output streams is done in the libavfilter chain (a
> scale filter is added to the end of the filterchain, one for each
> output video stream), the rescale/reformat context is used only to
> normalize the input video.

This means scaling videos twice if i understand it correctly

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The misfortune of the wise is better than the prosperity of the fool.
-- Epicurus
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20110427/3282bf4a/attachment.asc>