[FFmpeg-devel] Order of demuxed packets after a seek in an ogg container.

[wm4]

On Fri, 27 Jun 2014 21:24:14 +0200
Michael Niedermayer <michaelni at gmx.at> wrote:

> Hi
> 
> On Fri, Jun 27, 2014 at 07:07:32PM +0200, wm4 wrote:
> > On Wed, 25 Jun 2014 15:37:11 -0700
> > Dale Curtis <dalecurtis at chromium.org> wrote:
> > 
> > > Hi,
> > > 
> > > If av_seek_frame() is used to seek within a specific stream, are future
> > > av_read_frame() calls guaranteed to return all packets from all other
> > > streams in the container with timestamps after the seek timestamp?
> > 
> > I don't understand why this wouldn't be guaranteed. It sounds like a
> > quality of implementation issue with the ogg demuxer to me. Just
> > because it chose a simple way to implement this (apparently returning
> > all packets after the destination file position, according to the other
> > comments), it doesn't mean it's correct.
> > 
> > Just intuitively I'd say it should determine the target file position
> > for all active streams, then start demuxing from the lowest file
> > position of these streams, and skip any packets that are before the
> > seek target PTS.
> > 
> > No weird avformat_seek_file() API required, and no selecting of a
> > target stream either.
> > 
> > Am I wrong, am I missing something?
> 
> its not so simple, actually there are quite a few issues with using
> the av_seek_frame() API
> 
> The first issue with changing its behavior to consider multiple
> streams, is that its not documented to behave this way and thus
> breaks API which may break applications
> 
> Then theres the question of which streams to consider, the
> avformat_seek_file() API documents this and allows the user app to select
> the set of streams.
> With av_seek_frame() one could use the same choice or all streams but
> either way, the results could be very unexpected for the user
> application.
> For example consider an audio player which doesnt mark the video
> stream as discard it would mess up audio seek positions badly.

That audio player is obviously buggy for not disabling the video
stream, then.

> Or consider a video player and a file with multiple subtitle tracks
> the seek position would consider both video keyframes and subtitles
> while the application might not expect to get packets from 5 seconds
> before the video keyframe

I don't see how avformat_seek_file() solves any of this. You can't pass
a set of streams to consider to it. And I've never seen the behavior to
"snap" seeks to a subtitle stream if there's also an active audio or
video stream.

The semantics of seeking are pretty simple: jump to the first position
at which decoding can be resumed, that is _equal_to_ or _after_ the seek
target. Or if you can't resume at the exact timestamp, and the
AVSEEK_FLAG_BACKWARD flag is set, jump to the first position _before_
the seek target.

I don't know what's so unclear about these semantics - in fact, the
"new" API with its maximum and minimum PTS and an additional stream
parameter looks much more confusing to me.

> and theres the issue of seek direction in the av_seek_frame() API
> if a player is at time 100sec and seeks forward by 5sec using the
> audio stream but the video keyframe is before the 100sec mark
> you could end up seeking actually backward.

Only if the AVSEEK_FLAG_BACKWARD flag is set, and then you actually
want that it seeks backwards.

> The avformat_seek_file() API has a undoubtly funky looking description
> but it allows the user application to specify exactly in which area
> it wants the seek to go and documents that the result may be
> before this due to needing keyframes to allow presentation of the
> requested time
> 
> and then theres the speed issue, if av_seek_frame() would consider
> all streams by default, it would need more IO and more time
> which for applications which may or may not be what the application
> wants
> 
> Some of above could be hacked around with more flags to av_seek_frame()
> and some maybe be avoided by discarding packets wisely that are
> unneeded for presentation at the requested time
> but that still leaves the problem that a single target time and
> direction is not rich enough to specify the simple act of seek forward
> from position 100sec by 5 sec so that the point closest to 105 but
> clearly after 100sec is choosen

You might have a point, but actually it's not that useful in general. If
you want exact seeking, you'll have to skip data anyway, so you'd seek
to 105sec + BACKWARD flag set, and then skip everything until 105sec.

And for selecting streams: the only time I did that was to workaround
bugs in the subtitle seeking code. (Also why the heck are subtitle
formats not seekable with the "old" API?)