[FFmpeg-devel] Google Summer of Code participation

[Thilo Borgmann]

Vitor Sessak schrieb:
> Thilo Borgmann wrote:
>>
>>
>> Kostya schrieb:
>>> On Sat, Mar 28, 2009 at 12:20:32PM +0100, Thilo Borgmann wrote:
>>> [...]
>>>  
>>>> no. 12: "16-bit Interplay Video Decoder"
>>>> Sounds interesting and as there is a working 8-bit decoder, wich 
>>>> throws some errors if operating on the 16-bit demo file, there 
>>>> seems to be a good starting point. Unfortunately, the section about 
>>>> 16-bit opcodes is far from useful, if opcodes would have to be 
>>>> changed, this task becomes very difficult...
>>>> http://wiki.multimedia.cx/index.php?title=Interplay_Video
>>>>
>>>> -> The suggestion in the wiki is to ask the development community 
>>>> (you!) about a good start-off with that problem. So I do: Anyone 
>>>> who can help on this? Anyone with more information about the 16-bit 
>>>> mode?
>>>> (Already inspected the code here... sounds like altering the 
>>>> #define'd functions but how to know if it is 8-bit palette mode or 
>>>> 16-bit whatsoever mode?)
>>>>     
>>>  
>>> I suppose, it's only Google who knows more about this format.
>>> Maybe some container parameters should give demuxer a hint on 
>>> whether this
>>> is 16-bit mode or not.
>>>  
>>>  
>>>> no. 23: "CorePNG Decoder"
>>>> Well, the descriptions sound quite easy, BUT the current svn 
>>>> version of ffmpeg (as well as my very old one) say that it "could 
>>>> not find codec parameters". Thus, I suppose there is no existing 
>>>> PNG1 decoder wich decodes RGB I-Frame video? Is there a PNG1 coded 
>>>> RGB I-Frame demo video? So this task seems to me like implementing 
>>>> a whole decoder on top of the png image decoder?
>>>> http://samples.mplayerhq.hu/V-codecs/PNG1/
>>>>
>>>> -> Seems to be a not that difficult task but to have a complex 
>>>> start-off.
>>>>
>>>>
>>>> Anyone has an idea about these?
>>>>     
>>>
>>> I've looked at it once. If you use PNG decoder for that frames 
>>> you'll see
>>> good picture for the first frame and many colourful dots on the 
>>> latter. That's
>>> because its developer(s) decided to store latter frames as a 
>>> difference to
>>> the key frame and you have to use demuxer keyframe flag to 
>>> distinguish between
>>> both.
>>>
>>> Download http://samples.mplayerhq.hu/V-codecs/PNG1/corepng.avi
>>> run
>>> ffmpeg -i corepng.avi -f image2 -vcodec copy %04d.png
>>> and see that for yourself.
>>>
>>> P.S. We of FFmpeg welcome creativity since it's not that rare when 
>>> the only
>>> information about codec is located in libavcodec/*.c
>>>   
>>
>> Ok next problem arises. I'm currently trying to distinguish the I and 
>> P frames to be decoded.
>> As you mentioned, I tried to check the keyframe flag, that comes from 
>> 'outside'.
>> For that, I "av_log"ed the "key_frame" and "pict_type" attributes:
>>
>>  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> static int decode_frame(AVCodecContext *avctx,
>>                        void *data, int *data_size,
>>                        const uint8_t *buf, int buf_size)
>> {
>>    PNGDecContext * const s = avctx->priv_data;
>>    AVFrame *picture = data;
>>    AVFrame * const p= &s->picture;
>> #ifdef DEBUG
>> av_log(avctx, AV_LOG_DEBUG, "data->key_frame: %i, data->pict_type: 
>> %d\n",
>>       picture->key_frame, picture->pict_type);
>> av_log(avctx, AV_LOG_DEBUG, "s->picture->key_frame: %i, 
>> s->picture->pict_type: %d\n",
>>       p->key_frame, p->pict_type);
>> #endif
>>  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> (source listing is highly mailing list dependent - tell me how you 
>> like it if this is a pain in your eyes)
>>
>> As these debug outputs are telling that every frame is a key frame, 
>> which most of them are not, the question is if I'm looking at the 
>> wrong attributes or if this information is not parsed at all?
>
> There are three possibilities:
>
> 1- This information is coded somewhere in the AVI format (and thus it 
> is the work of the AVI demuxer to read it and set the parameters).
>
> 2- This information is in some reserved bits of the PNG stream (and so 
> it is in pngdec.c that it should be read).
>
> 3- This information is nowhere and the codec follow some stupid 
> convention like one frame every X are key-frames.
>
> I suggest you to look in the available documentation/source code to 
> know where this information is supposed to be. I find (1) unlikely 
> since ffmpeg AVI demuxer is not so buggy, and it would set 
> picture->key_frame correctly...
>
Found something here. I think, the avi dumexer does its job and reads 
the keyframe flags. I tested this by logging the PKT_FLAG_KEY or'ed into 
the 'flags' attribute of the 'AVPacket' structure (libavcodec/avidec.c: 
avi_read_packet)
This detected a keyframe at the 1st, 7th, 8th, 21st and 22nd frames of 
the stream, which makes sense as far as I can tell.

Ok, but unfortunately, the AVPacket structure is not passed completely 
to the decoder (libavcodec/pngdec.c: decode_frame()), just the 
attributes 'data' and 'size'. (attribute 'flags' hold the keyframe 
information, as I said above). These attributes are passed at the top 
level - ffplay.c: video_thread().

Next to this, an 'AVCodecContext' is passed to the decoder function. 
I've already tried to get the keyframe information out of them but they 
declare al frames to be keyframes. (see above for that)
As far as I understand the code now, this AVCodecContext is created once 
for each stream and therefore can not carry the information per frame. 
(This ist right, isn't it?)

Well, the keyframe flag is not coded in the PNG buffer either. So I do 
not see a chance to get the keyframe flag from the packages read by the 
demuxer into the decoder function. At least, not without changes at top 
level, which means at ffplay.c which is not an option in my eyes.

The last thing, CorePNG has a mode for a 'stupid' mode to declare every 
n-th frame as a keyframe but first, a generic decoding is better and 
should decode both modes as far as I got it by now, and second, the demo 
video is not using that stupid mode.

So, I wonder how other codecs get told about a keyframe they have to 
decode...

As you mentioned, the demuxer detects the keyframes but the 
picture->key_frame is not set. I checked this by my first attempt in 
reading ist value (was always set to 'keyframe') and by looking into the 
avidec.c (which is the avi demuxer, or am I completely wrong about 
that?), where the key_frame attribute is never touched.

Though, what would be the preferred way to carry this information from 
the package-struct to the decoder function? (I hope I just missed a clue 
somewhere but I looked through the code the whole day...)

TB