[FFmpeg-devel] [RFC] libavfilter audio API and related issues

[S.N. Hemanth Meenakshisundaram]

Stefano Sabatini <stefano.sabatini-lala <at> poste.it> writes:

> 
> Follow some notes about a possible design for the audio support in
> libavfilter.
> 
> AVFilterSamples struct 
> ======================
> 
> (Already defined in afilters, but renamed AVFilterBuffer at some
> point.)
> 
> Follows a possible definition (with some differences whit respect to
> that currently implemented in afilters):
> 
[...]
> 
> Audio/video synchronization
> ===========================
> 
> Some design work has to be done for understanding how request_samples()
> and request_frame() can work togheter.
> 
> I'm only considering ffplay for now, as it looks simpler than ffmpeg.
> 
> Currently audio and video follows two separate paths, audio is
> processed by the SDL thread thorugh the sdl_audio_callback function,
> while the video thread reads from the video queue whenever there are
> video packets available and process them.
> 

Currently, the dsl audio callback gets a decoded audio buffer via the
audio_decode_frame call and then seems to be doing AV sync via the
synchronize_audio call.

I was thinking about replacing this with the audio_request_samples function
suggested above. This is similar to what happens in video. The request_samples
would then propagate backwards through the audio filter chain until the input
audio filter (src filter) calls the audio_decode_frame to get decoded audio
samples and then passes them up the filter chain for processing.

Does this sound ok? Since the sdl_audio_callback will be making a
synchronize_audio call only after this, any additional delay introduced by
filtering would also get adjusted for.

> 
> Audio API
> =========
> 
> Much of what discussed in [3] is also relevant to libavfilter, and
> much of that functionality may be implemented in it if we decide to
> follow that route, e.g. audio mixing may be implemented in a filter.
> 
> Also it isn't clear where the audio API should be implemented,
> libavfilter will need to use the resampling functionality, if we don't
> want to make lavfi depend on lavc then this should be moved somewhere
> else (e.g. libavresample?).
> 
> References
> ==========
> 
> [1] afilters repo: svn://svn.ffmpeg.org/soc/afilters
> [2] gsoc discussion:
http://thread.gmane.org/gmane.comp.video.ffmpeg.soc/6163/focus=6178
> [3] http://wiki.multimedia.cx/index.php?title=FFmpeg_audio_API
> [4] http://wiki.multimedia.cx/index.php?title=Libavfilter
> [5]
http://wiki.multimedia.cx/index.php?title=FFmpeg_Summer_Of_Code_2010#Libavfilter_audio_work
> 
> ....
> 
> Comments are welcome.
> 
> Regards.

Regards,
Hemanth