[Mplayer-dev-eng] l3codeca.acm bug in VirtualAlloc - solved?

[Arpi]

Hi,

Thanks to Mw3 at IRC, I think we found the problem:

He got SIGILL for a file using the l3codeca.acm file
(that file is uploaded to thot, robots.avi. but it doesn't matter)

here is the gdb backtrace:

<Mw3> Program received signal SIGILL, Illegal instruction.
<Mw3> [Switching to Thread 1024 (LWP 25286)]
<Mw3> 0x4a0000 in strtold () from /lib/libc.so.6

<Mw3> #0  0x4a0000 in strtold () from /lib/libc.so.6
<Mw3> #1  0x4d0918 in malloc () from /lib/libc.so.6
<Mw3> #2  0x8078f3d in VirtualAlloc (address=0x400000, size=655360, type=12288
          , protection=64) at ext.c:473
<Mw3> #3  0x8082e19 in PE_LoadImage (handle=6, filename=0xbfffe8c8
          "/usr/lib/win32/l3codeca.acm", version=0xbfffe8c2) at pe_image.c:569

What's happen here?
libc is loaded to 0x4xxxxx and l3codeca.acm requested big VirtualAlloc to
that address. So libc's code has been overwritten with data bytes, so when it
executed malloc() call to libc it tried to run invalid code -> SIGILL

ldd ./mplayer:
<Mw3>         libc.so.6 => /lib/libc.so.6 (0x00460000)

Mw3's system:
<Mw3> Linux pange 2.4.3-ac13-getrewted #1 Fri Apr 27 01:09:04 CEST 2001 i686
libc: 2.2.2
cpu: celeron

----------------------------------

my system:
Linux arpi 2.4.3 #3 Mon Apr 2 20:08:49 CEST 2001 i686 unknown
libc: 2.1.3

arpi:/usr/src/linux# gcc -v
Reading specs from /usr/lib/gcc-lib/i386-slackware-linux/2.95.2/specs
gcc version 2.95.2 19991024 (release)

ldd ./mplayer:
        libc.so.6 => /lib/libc.so.6 (0x4026c000)

------------------------------------

So, for me everething is loaded above 0x40000000, while for Mw3 things are
mapped from 0x00000000. maybe the -ac kernels has some patch applied
to change default memory partitioning? Or it depends on libc version?

Any ideas?

We really has to fix mp3lib and make it default (l3codeca.acm is faster on
intel cpu's but has such problems)


A'rpi / Astral & ESP-team

--
mailto:arpi at thot.banki.hu
http://esp-team.scene.hu

_______________________________________________
Mplayer-dev-eng mailing list
Mplayer-dev-eng at lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/mplayer-dev-eng