[MPlayer-dev-eng] [PATCH] two trivial vbelib fixes

[Peter Kosinar]

Hello developers!

The following information is based on my limited knowledge of this area,
so my conclusions may be wrong.  In such case, I'd like to ask you to
correct me.

*** osdep/lrmi.c:run_vm86()
When calling lrmi_vm86(), the value of GS is saved (and later restored),
but the value of FS is not. This could interfere with some Windows-ish
DLLs (eg. QuickTime) which use addressing relative to FS [vm86() call
sets FS and GS to zero]. Fixed by copying the GS saving/restoring code.

*** osdep/vbelib.c:hide_terminal_output()
Buggy lines (XXX stands for stdin/stdout/stderr, Y is "r" or "w"):

%   my_XXX  = fopen(ttyname(fileno(XXX)),Y);

If XXX is not a tty, ttyname(fileno(XXX)) returns NULL, which is passed as
the first argument to fopen(). Usually, it would fail with EFAULT, because
0x0 is not a valid location. In this case, LRMI_init() was called before
and it mmap()ed first few bytes of /dev/mem (int. vectors, ...) to location
0x0. Thus, fopen() will open (or create, if Y is "w") a file with a strange
name (interrupt vector address, in my case "\x01").  Fixed by checking the
return value of ttyname() and acting appropriately.

Your sincerely,

Peter Kosinar
-------------- next part --------------
diff -Naur MPlayer-20031124/osdep/lrmi.c MPlayer-20031124-fixed/osdep/lrmi.c
--- MPlayer-20031124/osdep/lrmi.c	2003-11-13 21:53:40.000000000 +0100
+++ MPlayer-20031124-fixed/osdep/lrmi.c	2003-11-24 16:59:52.000000000 +0100
@@ -794,14 +794,16 @@
 	{
 	unsigned int vret;
 	sigset_t allsigs, cursigs;
-	unsigned long oldgs;
+	unsigned long oldgs, oldfs;
 
 	while (1)
 		{
 		sigfillset(&allsigs);
 	        sigprocmask(SIG_SETMASK, &allsigs, &cursigs);
 		asm volatile ("movl %%gs, %0" : "=g" (oldgs));
+		asm volatile ("movl %%fs, %0" : "=g" (oldfs));
 		vret = lrmi_vm86(&context.vm);
+		asm volatile ("movl %0, %%fs" :: "g" (oldfs));
 		asm volatile ("movl %0, %%gs" :: "g" (oldgs));
 		sigprocmask(SIG_SETMASK, &cursigs, NULL);
 
diff -Naur MPlayer-20031124/osdep/vbelib.c MPlayer-20031124-fixed/osdep/vbelib.c
--- MPlayer-20031124/osdep/vbelib.c	2003-11-13 21:53:40.000000000 +0100
+++ MPlayer-20031124-fixed/osdep/vbelib.c	2003-11-24 16:59:52.000000000 +0100
@@ -105,9 +105,10 @@
 /* TODO: do it only on LCD or DFP. We should extract such info from DDC */
 static void hide_terminal_output( void )
 {
-  my_stdin  = fopen(ttyname(fileno(stdin )),"r");
-  my_stdout = fopen(ttyname(fileno(stdout)),"w");
-  my_stderr = fopen(ttyname(fileno(stderr)),"w");
+  char *s;
+  s=ttyname(fileno(stdin )); my_stdin = s?fopen(s,"r"):NULL;
+  s=ttyname(fileno(stdout)); my_stdout= s?fopen(s,"w"):NULL;
+  s=ttyname(fileno(stderr)); my_stderr= s?fopen(s,"w"):NULL;
   __set_cursor_type(stdout,0);
 /*if(isatty(fileno(stdin ))) stdin =freopen("/dev/null","r",stdin );*/
   if(isatty(fileno(stdout))) stdout=freopen("/dev/null","w",stdout);