[FFmpeg-devel] [PATCH] avcodec/x86/hevc: fix luma 12b overflow

[Ronald S. Bultje]

On Sun, Feb 25, 2024 at 10:56 AM Ronald S. Bultje <rsbultje at gmail.com>
wrote:

> Hi,
>
> On Sun, Feb 25, 2024 at 3:28 AM J. Dekker <jdek at itanimul.li> wrote:
>
>> Weak filter can overflow in delta0 calculation before >> 4 in int16.
>>
>> Signed-off-by: J. Dekker <jdek at itanimul.li>
>> ---
>>
>>  I do not know x86 simd at all, so this is just an attempt to fix
>>  the implementation rather than write extremely performant code.
>>
>>  Suggestions welcome.
>>
>
> https://pastebin.com/KvcbQ2nK
>

Attached a slightly adjusted version which does sse2 in 16bit also.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-hevx-x86-deblock-fix-12bit-overflow.patch
Type: application/octet-stream
Size: 2698 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240225/5fffa5be/attachment.obj>