[FFmpeg-devel] Writing filters

[Michael Niedermayer]

On Wed, Jun 11, 2014 at 09:00:39PM +0200, James Darnley wrote:
> On 2014-06-11 17:40, Nicolas George wrote:
> > Le tridi 23 prairial, an CCXXII, Clément Bœsch a écrit :
> >> Yes, request_frame() is still a bit magic to me so I didn't feel
> >> comfortable explaining it.
> > 
> >> It's important to note that the API evolved quite a bit since the
> >> beginning. I *think* we can avoid request_frame in various cases nowadays
> >> with the help of FF_LINK_FLAG_REQUEST_LOOP.
> > 
> > It is really not that complicated once you understand that the whole process
> > is deeply recursive.
> > 
> > It all starts, for example, when the application asks a sink for a frame.
> > The sink calls request_frame() on its input filter, and the input filter
> > does the same, recursively, until it reaches a source. The source somehow
> > produces a frame and calls filter_frame() on its output, which does
> > the same, recursively, until it reaches the sink. Then everyone returns
> > success.
> 
> That does sound like a simple explanation.  In this case I would wonder
> why the filter_frame() chain is necessary.  I would have expected
> request_frame() to return a frame (in some fashion) which then gets
> processed.
> 
> (This is beginning to sound dangerously like bike-shedding.  I will just
> try to understand what exists.)
> 
> > Of course, it is not that simple.
> > 
> > The first issue is filters that require several frames to produce one, tile
> > for example. In that case, request_frame() has to loop until enough frames
> > are received to produce one. The framework is now capable of doing it: if
> > your filter requires just that, you can omit request_frame() and rely on the
> > default.
> > 
> > The second issue is filters with several inputs: consider concat versus
> > overlay, for example: only the filter knows what input needs a frame right
> > now in order to produce output. In that case, request_frame() can definitely
> > not be omitted, although it can rely on standard helpers for common cases.
> 
> This almost sounds like it reinforces what I thought above.  I would
> expect a filter's own request_frame() should just call its inputs'
> request_frame() functions as needed to make the frame.
> 
> I'm probably missing some considerations for random access, variable
> frame rates, interleaved audio-video packets, whatever.  (Again, I want
> to avoid arguing over the current design.)
> 
> On to an implementation question.
> 
> Let's say an Example filter needs several frames from its one and only
> input.  Example receives a call to example_request_frame() which then
> calls ff_request_frame() to get 1 frame from its input.  That "message"
> goes through to the input which then puts 1 frame into
> ff_filter_frame().  This eventually comes to Example's
> example_filter_frame().  This function knows it needs more frames to be
> able to do its work.
> 

> I assume that this function can finish/return without calling
> ff_filter_frame() itself, right?

yes


> 
> As I understand it, the return will go back though the functions to the
> input.  The input can then return from request_frame().  This return
> makes its way back to Example's example_request_frame().  The context
> struct for Example (or whatever mechanism might be used) shows that more
> input frames are required, so example_request_frame() will call
> ff_request_frame() again.

yes


> 
> Is the reason for the do { ff_request_frame() } while(); loops in
> showwaves and showspectrum?  Frames are requested until filter_frame()
> marks that it is done?
> 

> On the other side, if a filter makes 2 frames from every input frame,
> can it call ff_filter_frame() twice from its own filter_frame()?

yes, though its nicer if it waits for a request_frame() call
or the next filter_frame if theres no request_frame
as this can spread processing time from subsequent filters over
more frames instead of having spikes of processing time at the
frames which are duplicated
Nicolas might have other arguments here, i didnt think too deeply
about this now


> 
> On the first call to ff_filter_frame() the "returns" come back to this
> filter, then it makes its second call to ff_filter_frame.
> 
> 



> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Complexity theory is the science of finding the exact solution to an
approximation. Benchmarking OTOH is finding an approximation of the exact
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20140619/9999705c/attachment.asc>