[FFmpeg-devel] [FFmpeg-user] [PATCH] Fix seg fault on 64bit x86 and h264 video decode.

[James Courtier-Dutton]

Patch and example media stream in:
 /MPlayer/incoming/jcd

I reproduce the problem by using either mplayer or xine and the latest
ffmpeg svn.

Kind Regards

James


On 22 January 2010 23:59, Baptiste Coudurier
<baptiste.coudurier at gmail.com> wrote:
> On 01/22/2010 03:23 PM, James Courtier-Dutton wrote:
>>
>> 2010/1/22 Baptiste Coudurier<baptiste.coudurier at gmail.com>:
>>>
>>> Hi,
>>>
>>> On 01/22/2010 03:06 PM, James Courtier-Dutton wrote:
>>>>
>>>> Hi,
>>>>
>>>> This patch fixes a seg fault when playing h264 video that was created
>>>> by a Canon HD camcorder.
>>>> It seg faults at lines with this in:
>>>> beta ?= (beta_table+52)[qp + h->slice_beta_offset];
>>>>
>>>> qp can be 32 bit negative number. On a 32bit system, this would just
>>>> wrap round and act like a signed value.
>>>> On a 64bit system, the 32 bit negative number looks like a very large
>>>> positive number. This results in a seg fault when dereferenced.
>>>>
>>>> Replacing the above with something like the following fixes the
>>>> problem on both 32bit and 64bit systems:
>>>> int index_b = 52 + qp + h->slice_beta_offset;
>>>> beta = beta_table[index_b];
>>>
>>> Thanks a lot for the patch, can you please resend it to ffmpeg-devel
>>> mailing
>>> list ?
>>>
>>> Also please attach a sample to reproduce the issue.
>>>
>>> Thanks a lot for your help.
>>>
>>
>> I have requested to join the ffmpeg-devel list twice, but it fails to
>> send me a confirmation email.
>> I then tried ffmpeg-user and that worked, so I could at least post the
>> patch.
>> The sample file is 52Megs. It is an HD Video file, that is why it is
>> so big. Where should I put it?
>
> Instructions are at http://www.ffmpeg.org/bugreports.html
>
> You can still send a mail to ffmpeg-devel ml, it will get moderated but
> approved pretty quickly hopefully.
>
> Thanks for your help.
>
> --
> Baptiste COUDURIER
> Key fingerprint ? ? ? ? ? ? ? ? 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA
> FFmpeg maintainer ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?http://www.ffmpeg.org
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at mplayerhq.hu
> https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
>