[FFmpeg-devel] libavfilter API design in a realtime environment

[Nicolas George]

Le quartidi 4 floréal, an CCXXIV, Paul B Mahol a écrit :
> This is really, really bad. Frame do should have duration. And audio
> ones do have.

I used to think that, but after more careful thought I no longer do.

Audio frames have a number samples, they can not work without and that gives
us the duration. It makes some of the problems I will indicate go away, but
not all of them.

There are two sides to the problem.

First, cur_frame.pts, cur_frame.duration and next_frame.pts have a
consistency constraint. Consistency constraints are always trouble, they are
an endless source of bugs because there will always be part of code that
update one field without updating the other.

Second, the duration is often not available immediately, and deriving it is
burdensome.

A few more specific examples:

What happens if the duration and timestamps do not match? Some filters will
behave one way, some filters will behave another. Filters that trust the
duration may accumulate rounding errors if the timebase is not chosen
correctly (think matroska timebase 1/1000 versus NTSC frame rate), etc.

Is the duration mandatory? If it is not, then all filters must be able to
work without it, and we are back to square one. If it is, what happens when
it is not available? Who is responsible for computing it?

A lot of formats do not store the duration. And even when they do, it is not
always reliable. Should the application demux another frame just to fill it?

What about the select filter? In order to compute the duration, it would
essentially have to process the next frames until one is selected.

More generally, in order to reduce the latency in a few cases (filters that
use framesync), you would have to increase the latency at a lot more places.

Much simpler to not have a duration. Filters that really need it wait an
extra frame, that is not a problem.

Regards,

-- 
  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20160422/58e66264/attachment.sig>