[FFmpeg-devel] [PATCH 3/3] swscale/swscale_unscaled: fix packed16togbra16() for formats with bpc between 9-14 bits

[Ramiro Polla]

On Sun, May 18, 2025 at 11:17 PM James Almer <jamrial at gmail.com> wrote:
> On 5/18/2025 6:14 PM, James Almer wrote:
> > On 5/18/2025 5:52 PM, Ramiro Polla wrote:
> >> Currently, packed16togbra16() always sets the alpha value to 0xFFFF,
> >> without taking the bit depth into consideration.
> >>
> >> This commit restricts the alpha value to the bit depth.
> >
> > packed16togbra16() seems to only be called for BGR48 and BGRA64, both of
> > which are 16bits, so this change is superfluous.
>
> Ah, nevermind, i was looking at the src formats, not dst.
>
> Are there no tests that cover these paths? I added a bunch a couple
> months ago, so maybe it could be extended.

It can be reproduced with:
./libswscale/tests/swscale -unscaled 1 -src xyz12le -dst gbrap12be

A little bit more information: this bug only happens on x86. The
problem arises from the optimized conversion that comes afterwards,
from gbrap12be to yuva444p, in ff_hscale14to15_4_ssse3(). It has
something to do with pmaddwd not working on unsigned values IIRC. We
could fix ff_hscale14to15_4_ssse3() to also work correctly with 0xFFFF
on bit depths < 16, or we could just not write 0xFFFF there in the
first place, which is what this patch does.

I thought about adding libswscale/tests/swscale to FATE, but the tests
take way too long.

Ramiro