[Ffmpeg-devel] segfault in ff_mpeg1_find_frame_end()

[Stefan Lucke]

Hi,

We are developing a plugin for vdr named softdevice. Plugins homepage
is at http://softdevice.berlios.de .

I get a segfault in function mentioned in subject. The segfault happens
when there is a transition in aspect ratio from 16:9 to 4:3. It happens not
when the stream is played the first time but when its played the 3rd time.
But this may vary (segfault may happen at the first time too).

To ensure that it has nothing to with PADDING bytes I added extra
code that mallocs and copies data before decoding and zeros
from "data+size" FF_INPUT_BUFFER_PADDING_SIZE bytes.

I'm using ffmpeg cvs version from today.

What could be done to track down the cause of segfault?

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 655401 (LWP 4847)]
0x4046d1d1 in ff_mpeg1_find_frame_end () from /usr/local/lib/libavcodec.so
(gdb) bt
#0  0x4046d1d1 in ff_mpeg1_find_frame_end () from /usr/local/lib/libavcodec.so
#1  0x4046c8a9 in ff_mpeg1_find_frame_end () from /usr/local/lib/libavcodec.so
#2  0x40373154 in avcodec_decode_video () from /usr/local/lib/libavcodec.so
#3  0x403099f9 in cVideoStreamDecoder::DecodePacket(AVPacket*) (this=0x841f820, pkt=0x8423a24) at mpeg2decoder.c:497
#4  0x40308b29 in cStreamDecoder::Action() (this=0x841f820) at mpeg2decoder.c:199
#5  0x080dbc40 in cThread::StartThread(cThread*) (Thread=0x841f820) at thread.c:227
#6  0x40059f60 in pthread_start_thread () from /lib/i686/libpthread.so.0
#7  0x4005a0fe in pthread_start_thread_event () from /lib/i686/libpthread.so.0
#8  0x40270327 in clone () from /lib/i686/libc.so.6

and the following at the first try when using  statically linked ffmpegs libs:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 245776 (LWP 5433)]
mpeg1_decode_picture (avctx=0x8368fc8, buf=0x86dc258 <Address 0x86dc258 out of bounds>, buf_size=18921) at bswap.h:30
30      {
(gdb) bt
#0  mpeg1_decode_picture (avctx=0x8368fc8, buf=0x86dc258 <Address 0x86dc258 out of bounds>, buf_size=18921) at bswap.h:30
#1  0x4045a799 in mpeg_decode_frame (avctx=0x8368fc8, data=0x8451340, data_size=0x489dfe54, buf=0x86dc190 <Address 0x86dc190 out of bounds>,
    buf_size=19121) at mpeg12.c:3023
#2  0x40364fc4 in avcodec_decode_video (avctx=0x8368fc8, picture=0x8451340, got_picture_ptr=0x489dfe54, buf=0x8432180 "", buf_size=2039) at utils.c:593
#3  0x4035f199 in cVideoStreamDecoder::DecodePacket(AVPacket*) (this=0x8440b48, pkt=0x8444d4c) at mpeg2decoder.c:497
#4  0x4035e2c9 in cStreamDecoder::Action() (this=0x8440b48) at mpeg2decoder.c:199
#5  0x080dbc40 in cThread::StartThread(cThread*) (Thread=0x8440b48) at thread.c:227
#6  0x40059f60 in pthread_start_thread () from /lib/i686/libpthread.so.0
#7  0x4005a0fe in pthread_start_thread_event () from /lib/i686/libpthread.so.0
#8  0x40270327 in clone () from /lib/i686/libc.so.6

-- 
Stefan Lucke