[MPlayer-dev-eng] [PATCH] RGB/BGR bug in yuv2rgb_altivec.c

[Marcus Comstedt]

Hi.

I found a bug i yuv2rgb_altivec.c.  The thing is that the macros
out_rgb24() and out_bgr24() will actually _both_ output BGR!
The reason for this is that vec_merge3() has the in parameters
x0,x1,x2 declared in the wrong order.  Thus vec_mstrgb24(), which
calls it with the parameters in the order x0,x1,x2, will have the
colors swapped and actually generate BGR.  Conversely, vec_mstbgr24()
which calls vec_merge() with the parameters in the x2,x1,x0 order will
_not_ have the colors swapped, and output RGB!  The last problem is
compensated for in the out_bgr24() macro by again swapping the order
of the parameters though.  However, out_rgb24() does not swap the
parameters and thus gives incorrect output.

Ok, so my proposal for how to clean up this mess is the following:

* Change vec_merge3() to take the parameters in the order x0,x1,x2,
  to avoid further confusion.

* Remove vec_mstbgr24(), and simply have out_bgr24() call
  vec_mstrgb24() with the parameters swapped, like it is done for the
  32 bit variants.

I'm attaching a patch to perform these changes.


  // Marcus


-------------- next part --------------
A non-text attachment was scrubbed...
Name: yuv2rgb_altivec.c.patch
Type: text/x-patch
Size: 1205 bytes
Desc: Patch to postproc/yuv2rgb_altivec.c fixing RGB24 output	problem.
URL: <http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/attachments/20041031/6a13c43f/attachment.bin>