[Ffmpeg-devel] avcodec_find_best_pix_fmt1 matching problem

[Panagiotis Issaris]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Panagiotis Issaris schreef:
> Isn't there a problem on line 677 of this code?
> 
> It appears to me that PIX_FMT_NB is to large (being 37), to allow the
> bitshifting to work flawlessly.
> 
>      666 static int avcodec_find_best_pix_fmt1(int pix_fmt_mask,
>      667                                       int src_pix_fmt,
>      668                                       int has_alpha,
>      669                                       int loss_mask)
>      670 {
>      671     int dist, i, loss, min_dist, dst_pix_fmt;
>[...]
>      676     for(i = 0;i < PIX_FMT_NB; i++) {
>      677         if (pix_fmt_mask & (1 << i)) {
>      678             loss = avcodec_get_pix_fmt_loss(i, src_pix_fmt,
> has_alpha) & loss_mask;
>[...]

The attached patch tries to fix the problem. It does change the
signature of one of the public API functions though.
avcodec_find_best_fix_fmt() which calls the above function wasn't used
anywhere in the FFmpeg sourcetree, but ofcourse it might have been used
by others.

With friendly regards,
Takis
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFF5qep9kOxLuzz4CkRAtsuAJwN5Ku/9Hw7f38t+t2oziylluesLgCdGY90
Ux47QySCo82qqBKAawnjLoU=
=+SoR
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pi-20070301T110048-ffmpeg-find_best_pix_fmt1_fix.diff
Type: text/x-patch
Size: 1861 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20070301/2126b926/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pi-20070301T110048-ffmpeg-find_best_pix_fmt1_fix.diff.sig
Type: application/octet-stream
Size: 65 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20070301/2126b926/attachment.obj>