[FFmpeg-devel] Fixing av_seek

[Don Moir]

> Finnally getting around to looking more closely at seek problems in 
> ffmpeg.
>
> This is a follow up to ticket #504. 
> https://ffmpeg.org/trac/ffmpeg/ticket/504
>
> My goal is to write a higher level seek function that depends on the lower 
> level ffmpeg seek functions. In some cases, patches to ffmpeg may need to 
> be applied.
>
> The higher level function will only deal with a timestamp that might 
> correspond to a slider or similiar, and will seek very close to that 
> timestamp without image distortion.
>
> There are several categories of success or failure when seeking. This 
> first group (Group A), deals with things that work or are easy to deal 
> with.
>
> o - everything works and works well. There are plently of index_entries 
> and the format in question does the right thing. CODEC_ID_FLV1 is an 
> example that works well but some other formats work just as well.
>
> o - there are no index_entries or none of the index_entries are marked as 
> AVINDEX_KEYFRAME. This will cause avformat_seek_file or av_seek_frame to 
> fail. If there are index_entries and the first entry represents the first 
> frame, then I have found that just marking this first index_entry with 
> AVINDEX_KEYFRAME fixes the ffmpeg seek failure problem. If there are no 
> index_entries then you can add one using av_add_index_entry using the 
> correct parameters for the first frame and marking it as AVINDEX_KEYFRAME. 
> This assumes the first frame must be a keyframe and easier to deal with 
> this at startup rather then trying to deal with it when doing a seek.
>
> o - lower level seek functions mostly deal with keyframes and thats all 
> good, but thats not good enough for an end user trying to use a slider to 
> seek. So get as close as possible using one of the ffmpeg seek functions 
> and then proceed forward to the timestamp of interest. This works pretty 
> good and is fast enough. Mostly, good files will have enough 
> AVINDEX_KEYFRAME index_entries to make this behave quite well. Even when 
> there are not enough entries, it works good. You probably want to do this 
> in a separate thread. Make sure you first seek to a timestamp that is less 
> than or equal to the timestamp of interest so you can proceed forward. 
> There are probably ways to improve on this.
>
> The second group (Group B) falls into the class of failures or bad 
> behavior that is not easy to deal with. e.g. an inability to make it work 
> as expected for various reasons.
>
> I am still dealing with trying to find out Group B problems. I have had 
> issues with MPEG1, THEORA, and a few others, and I will know more about 
> these problems as I get into it further.
>
> I use VLC and SMPlayer to cross check and both of these have their share 
> of seek problems. I use my own player to test with.

Using this h264 file, the index_entries are built on the fly and appear to 
be incorrect.

http://sms.pangolin.com/temp/bad_seek_index_h264.zip  (2.5mb)

codebase: git-67f5650 - after fixes with h264 seeking but I don't see any 
difference with older codebase versions than this one.

Just after av_open_input_file, there are 2 index_entries. The first one has 
timestamp of 0 and 2nd has timestamp of 0x48f. The odd thing is they both 
have the same pos which 0x122F. So seems strange from the start. As playback 
progresses, the number of index_entries increases and this is normal.

When you try to seek on this file to a supposed keyframe, you don't get a 
finished frame until much later (~4 seconds). What should happen is if you 
have seeked to a keyframe, you should be getting a finished frame 
immediately. This is the way I see it at any rate. So trying to seek into 
this file breaks the playback, meaning you will get a long pause before 
normal playback resumes because you are not getting a finsihed frame when 
you should be. If I just seek to zero on this file, then all is good and I 
get the finished frames as expected but of course it's slower to reach the 
requested timestamp.

I bring this up because I think this represents a key problem with some of 
the codec types. That is index_entries are added that don't click with 
actual keyframes. I asked a question about this relationship on the original 
ticket but there was no response.

I think getting this fixed will go a long way in fixing several seek 
problems. But I need information from you if you have any on this. If you 
don't have a answer, maybe "I or we don't know" would be better than no 
answer. Also not to just focus on h264 here because I believe there are 
other formats with this very same problem.