[Ffmpeg-devel] av_seek_frame() units

[Steve Willis]

Gary Corcoran wrote:
> Steve Willis wrote:
> 
>> Gary Corcoran wrote:
>>
>>> Steve Willis wrote:
>>>
>>>> Nathan Kurz wrote:
>>>>
>>>>> On Wed, Jul 06, 2005 at 12:04:44PM -0700, Ian Gowen wrote:
>>>>>
>>>>>> On 7/6/05, Andy Parkins <andyparkins at gmail.com> wrote:
>>>>>>
>>>>>>> I'm not sure why you would need to do this.  What source for 
>>>>>>> frame_number are
>>>>>>> you using - it's not in any of the AV structures.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> I'm working on a video editing application. If the user wants to
>>>>>> preview frame 320, I
>>>>>> need it to seek to frame 320, not the nearest keyframe or a frame in
>>>>>> the general
>>>>>> vicinity of frame 320.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Not to be too discouraging, but you should also know that seek is
>>>>> badly broken for many formats.  It's quite possible that you are doing
>>>>> things correctly, but are being bitten by bugs.  If you search  the
>>>>> archives from (I think) the end of last year you'll find a fairly
>>>>> elaborate test program I wrote to test which formats work.  I had
>>>>> submitted it as a regression test, but as it pointed out new bugs
>>>>> rather than regressions it wasn't included in the test suite.
>>>>>
>>>>> Nathan Kurz
>>>>> nate at verse.com
>>>>>
>>>>> _______________________________________________
>>>>> ffmpeg-devel mailing list
>>>>> ffmpeg-devel at mplayerhq.hu
>>>>> http://mplayerhq.hu/mailman/listinfo/ffmpeg-devel
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> It sounds like Ian and I are both working on applications that need 
>>>> precise frame positioning. In light of what Nathan said above, can 
>>>> anyone offer recommendations for the best way to ensure that the 
>>>> user is looking at the frame he or she thinks they should be looking 
>>>> at? Is there a reliable way to detect when seek has gone wrong ("I 
>>>> expected to be looking at the frame for time 13.125 seconds, but I 
>>>> actually got the frame for time X...error!")?
>>>>
>>>> In response to my original question a couple of weeks ago, someone 
>>>> suggested using a format like MJPEG that treats all frames as 
>>>> keyframes. If there is not a more general solution for precicely 
>>>> positioning arbitrarily-formatted streams, could someone give me an 
>>>> expert opinion on the best format to transcode all input videos 
>>>> into? The videos I am dealing with are printed from high speed film 
>>>> (1000 fps) for scientific analysis frame-by-frame. I'm looking for a 
>>>> reasonable balance between decoding speed and size. Image quality is 
>>>> of course important for seeing detail in each frame of interest. 
>>>> Most important is that the frame shown is actually correct for the 
>>>> time in question to match up with external sensor data, so a format 
>>>> that does not use keyframes would also be important.
>>>
>>>
>>>
>>>
>>> Couldn't you convert it to MPEG, specifying that only I-frames 
>>> (keyframes)
>>> be used?  Then the "seek only directly works to I-frames" feature 
>>> wouldn't
>>> be a problem...
>>
>>
>>
>> I don't think MPEG supports arbitrary frame rates. The FAQ on the 
>> ffmpeg Web site mentions that only a small number of frame rates are 
>> allowed. The videos are printed from film. They are usually in the 
>> ballpark of 1000 fps. No player can realistically play them at this 
>> speed, but they work very well for frame-by-frame analysis. One other 
>> requirement for a format I should mention is the ability to encode an 
>> arbitrary number of video streams. Some of these videos contain 
>> multiple video tracks with different angles of the same event. I need 
>> a format that lets me preserve this in a single file.
>>
>> Thanks for your suggestion, and please, let me know if I'm wrong about 
>> MPEG and framerates!
> 
> 
> Well, since it sounds like you do NOT require realtime playback of the
> video - you want to edit and/or look frame-by-frame, right? - then who
> cares what framerate the MPEG is?  That is, couldn't you just convert
> every frame of your original video to MPEG at say 30 fps, and then
> edit/examine frames?  Sure if you tried to play it realtime it would
> be 33 times too slow - but wouldn't that be an advantage for scientific
> analysis?  :)  Then after editing you could always convert to some other
> format that supported 1000fps (but isn't seekable), if that's needed.
> 
> But I don't know anything about encoding multiple video streams in
> a single file...

Of course you are right. That was stupid of me.

As to encoding multiple streams, I believe MPEG-2 is an excellent 
choice. If I recall my course in multimedia information systems, MPEG-2 
improved upon the 3-stream model of MPEG-1 (video, audio, system) by 
allowing multiple streams of arbitrary type to be interleaved. I'll have 
to verify this, though (comments or corrections are much appreciated.)

If I can't find a reliable way to position media in other formats to 
precise frames, I'll probably go this route. I don't know if this helps 
Ian, though. For editing, he probably won't want the quality loss of 
converting to MPEG (lossy) then back to another format. Does anyone have 
any insight on how "professional" video editing programs like Adobe 
Premiere allow users to treat any frame as a keyframe?

Steve