[FFmpeg-devel] [RFC] Seeking API

[Baptiste Coudurier]

Michael Niedermayer wrote:
> On Thu, Jan 22, 2009 at 02:42:04PM -0800, Baptiste Coudurier wrote:
>> Michael Niedermayer wrote:
>>> On Thu, Jan 22, 2009 at 12:46:42PM -0800, Baptiste Coudurier wrote:
>>>> Hi Michael,
>>>>
>>>> Michael Niedermayer wrote:
>>>>> Hi
>>>>>
>>>>> Do we need a new API for seeking?
>>>>> I think we do
>>>>>
>>>>> * currently seeking happens based on >=X or <=X this is not truely ideal
>>>>>   for a normal player, for example at position 100sec and seeking 10sec
>>>>>   forward we would prefer 109 over 200sec likely but not 99sec over 200sec
>>>>> * seeking in relation to a single specific stream makes little sense, rather
>>>>>   seeking should happen relative to the set of streams that is presented
>>>>>   to the user (= the ones not disabled by AVStream.discard)
>>>>>
>>>>> Are there other issues? requirements?
>>>> Yes, I'd like a new AVSEEK_FLAG_FRAMENUM for containers supporting it
>>>> (mov/mp4/r3d)
>>> This also needs a stream_index  and could be done by the user app with
>>> ts= st.index_entries[frame_num].timestamp;
>> True, but aren't we defining an API to simplify _usage_ ?
> 
> yes but wont many programs that seek by frame num want/mess with AVIndex
> anyway?

I agree in principle, also they would probably want to mess with the
smallest amount of information so if av_seek_frame provides the feature,
they would see that seeking can be done by frame num and know how,
thanks to the documentation :>

>> Also we could handle FRAMENUM | NEXT_KEYFRAME or PREV_KEYFRAME with the
>> help of min_distance value, which becomes complicated for the user.
>>
>> Needing a stream_index is ok if documented IMHO.
> 
> I dont mind adding FRAMENUM & stream_index but this will make the API
> more complex. see:
> 
> /**
>  * Seek to timestamp ts.
>  * Seeking will be done so that the point from which all active streams
>  * can be presented successfully will be closest to ts and within min/max_ts.
>  *
>  * if flags contain AVSEEK_FLAG_BYTE then all timestamps are in byte and
>  * are the file position (this may not be supported by all demuxers).
>  * if flags contain AVSEEK_FLAG_FRAME then all timestamps are in frames
>  * in the stream with stream_index (this may not be supported by all demuxers).
>  * else all timestamps are in units of the stream selected by stream_index or
>  * if its -1 AV_TIME_BASE units.
>  * if flags contain AVSEEK_FLAG_ANY then non keyframes are treated as
>  * keyframes (this may not be supported by all demuxers).
>  *
>  * @param stream_index index of the stream which is used as timebase reference.
>  * @param min_ts smallest acceptable timestamp
>  * @param ts target timestamp
>  * @param max_ts largest acceptable timestamp
>  * @param flags flags
>  * @returns >=0 on success, error code otherwise
>  */
> int av_seek_frame2(AVFormatContext *s, int stream_index, int64_t min_ts, int64_t ts, int64_t max_ts, int flags);

I like it, are min/max_ts allowed to be 0/-1/AV_NOPTS_VALUE ? What
happens in this case ? No limits ?

> and this still skiped one issue, namely that AVSEEK_FLAG_FRAME will with this
> API not seek to the wanted frame unless the active set selected by
> AVStream.discard is just that single stream.
> We can add yet another flag to override this of course ...

Could we always allow to specify a discarded stream and take it into
account nonetheless ? What harm could it cause ?

Should we let the possibility to reeanable a stream ? In this case
populating index for all streams is useful.

Atm, it seems entries are added even if stream is discarded, at least
mov and mpeg ps.

I'd say -1 with FLAG_FRAME could mean first video track for ease of use,
it's not really importand though.

>>>> Not sure if this is much related to seeking:
>>>> I'd like containers to export the AVIndex feature usage and add pts to
>>>> AVIndexEntry, also a way to generate the full AVIndexEntry and have the
>>>> possibility to export it as file to be loaded later.
>>>>
>>>> Also user needs to specify if he wants to seek by dts or pts and
>>>> therefore what timestamp value refers to.
>>> Is there any use case in which the user wants to seek by dts?
>> Not sure, currently, mpeg ps demuxer add dts to index entries, same for mov.
>> So either you add pts to index entries but with reordering, binary
>> search is difficult.
> 
> seeking in many demuxers will need to be reworked if we want to improve
> things.

Definitely, what is the plan to keep compatibility with av_seek_frame ?

> also dts&pts in mpeg-ps only have to be stored once every 0.5 sec or so
> which is a bigger issue than reordering.

Right, this should be handled also.

>> Or you add dts and export first_dts value, so it is possible to seek to
>> first dts.
>>
> 
>> In all case "timestamp" value needs clarification.
> 
> all timestamps where intended to be presentation timestamps
> i felt that:
>     "Seeking will be done so that the point from which all active streams
>     can be presented successfully will be closest to ts and within min/max_ts."
> was makeing that clear
> do you have some suggestion to improve this?
> i could rename ts to pts ...

Using pts would make it clear and avoid any confusion for future
demuxers, I believe.

So should demuxers be changed to add pts in index ?
av_index_search_timestamps should also be changed then.

Should we add pts to AVIndexEntry, to have dts and pts so simplify
search, or is it unnecessary ?

>> Also "pos" needs clarification, I propose to use pos as position which
>> enable byte seeking to and then call av_read_frame, for containers
>> supporting byte seeking (mpeg ps, mpeg ts), with rules like next
>> containing access unit.
> 
> patch welcome

Ok.

>> Containers should export the information when supporting byte seeking
>> (mov/mp4 does not).
> 
> yes, pkt.pos should be required when byte seeking is supported.
> doc patch welcome as well
> 
> ( i think little harm is done if such doc stuff is commited already before
>   the actual implementation, would simplify team work )

Yes, I think so.

-- 
Baptiste COUDURIER                              GnuPG Key Id: 0x5C1ABAAA
Key fingerprint                 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA
checking for life_signs in -lkenny... no
FFmpeg maintainer                                  http://www.ffmpeg.org