[FFmpeg-devel] [PATCH 2/5] lavfi/gradfun: fix rounding in MMX code.

[Loren Merritt]

On Fri, 7 Dec 2012, Reimar Döffinger wrote:
> On 7 Dec 2012, at 16:28, Loren Merritt <lorenm at u.washington.edu> wrote:
>> On Fri, 7 Dec 2012, Reimar Döffinger wrote:
>>> "Clément Bÿÿsch" <ubitux at gmail.com> wrote:
>>>
>>>> Current code divide before increasing precision.
>>>
>>> Are you sure the shift can _never_ overflow? It is definitely very tight.
>>
>> I expect it can overflow, but the case where it does so also has m=0 for
>> sane filter strengths. So the appropriate fix would be to limit strength,
>> not change the asm.
>
> Since the proposal is to change the asm, do you mean the patch is right, it just should be combined with limiting the strength?

Yes.

> I have no objections about that, I just kind of assumed there was a good reason for doing things in this order.

The original order probably was about avoiding overflow, but limiting
strength is better.

--Loren Merritt