[FFmpeg-devel] Writing filters

[James Darnley]

On 2014-06-11 17:40, Nicolas George wrote:
> Le tridi 23 prairial, an CCXXII, Clément Bœsch a écrit :
>> Yes, request_frame() is still a bit magic to me so I didn't feel
>> comfortable explaining it.
> 
>> It's important to note that the API evolved quite a bit since the
>> beginning. I *think* we can avoid request_frame in various cases nowadays
>> with the help of FF_LINK_FLAG_REQUEST_LOOP.
> 
> It is really not that complicated once you understand that the whole process
> is deeply recursive.
> 
> It all starts, for example, when the application asks a sink for a frame.
> The sink calls request_frame() on its input filter, and the input filter
> does the same, recursively, until it reaches a source. The source somehow
> produces a frame and calls filter_frame() on its output, which does
> the same, recursively, until it reaches the sink. Then everyone returns
> success.

That does sound like a simple explanation.  In this case I would wonder
why the filter_frame() chain is necessary.  I would have expected
request_frame() to return a frame (in some fashion) which then gets
processed.

(This is beginning to sound dangerously like bike-shedding.  I will just
try to understand what exists.)

> Of course, it is not that simple.
> 
> The first issue is filters that require several frames to produce one, tile
> for example. In that case, request_frame() has to loop until enough frames
> are received to produce one. The framework is now capable of doing it: if
> your filter requires just that, you can omit request_frame() and rely on the
> default.
> 
> The second issue is filters with several inputs: consider concat versus
> overlay, for example: only the filter knows what input needs a frame right
> now in order to produce output. In that case, request_frame() can definitely
> not be omitted, although it can rely on standard helpers for common cases.

This almost sounds like it reinforces what I thought above.  I would
expect a filter's own request_frame() should just call its inputs'
request_frame() functions as needed to make the frame.

I'm probably missing some considerations for random access, variable
frame rates, interleaved audio-video packets, whatever.  (Again, I want
to avoid arguing over the current design.)

On to an implementation question.

Let's say an Example filter needs several frames from its one and only
input.  Example receives a call to example_request_frame() which then
calls ff_request_frame() to get 1 frame from its input.  That "message"
goes through to the input which then puts 1 frame into
ff_filter_frame().  This eventually comes to Example's
example_filter_frame().  This function knows it needs more frames to be
able to do its work.

I assume that this function can finish/return without calling
ff_filter_frame() itself, right?

As I understand it, the return will go back though the functions to the
input.  The input can then return from request_frame().  This return
makes its way back to Example's example_request_frame().  The context
struct for Example (or whatever mechanism might be used) shows that more
input frames are required, so example_request_frame() will call
ff_request_frame() again.

Is the reason for the do { ff_request_frame() } while(); loops in
showwaves and showspectrum?  Frames are requested until filter_frame()
marks that it is done?

On the other side, if a filter makes 2 frames from every input frame,
can it call ff_filter_frame() twice from its own filter_frame()?

On the first call to ff_filter_frame() the "returns" come back to this
filter, then it makes its second call to ff_filter_frame.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 683 bytes
Desc: OpenPGP digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20140611/a3468940/attachment.asc>