[FFmpeg-devel] [PATCH 2/5] fate: avoid framemd5, use framecrc its faster

[Reimar Döffinger]

On 09.05.2013, at 13:14, Michael Niedermayer <michaelni at gmx.at> wrote:
> On Thu, May 09, 2013 at 07:58:45AM +0200, Reimar Döffinger wrote:
>> On 09.05.2013, at 02:12, Michael Niedermayer <michaelni at gmx.at> wrote:
>>> Ive never seen crc or adler fail on real world data. It sure does
>>> happen, for example you should find a colission in 100000 random files
>>> but thats not the question.
>> 
>> If it was random files I would have no concerns...
>> 
>>> If i safe 10minutes a day, that sure is alot more than i will ever
>>> spend on debuging crc & adler32 colission related bugs.
>>> and i can spend these 10minutes on debuging other bugs thus ...
>> 
>> I don't really believe that FATE being 10 seconds faster would really safe you any time at all in the end, which might be part of why I have doubts about this making sense, sorry for being such a sceptic :-)
> 
> please pick 50 activities you do each day and on each wait 10
> seconds each day doing nothing. ... that is if thats irrelevant and no
> problem for you

Well, usually I don't sit there and wait while tests run, that's why I didn't really think it would matter. But I already said sorry, don't get mad at me.

>>>> It also suggests that they may fail to detect e.g. padding changing
>>>> from 0x00 to 0xFF. This might be relevant to the valgrind memory
>>> 
>>> do you have a example of such colission with changed padding ?
>> 
>> Paper said it can't distinguish long runs of those since they are both 0 in one's complement, but I don't feel like doing my own research.
> 
> are you talking about fletcher or adler ?

They wrote both are based on one's complement and thus have the same issue.

>> Well, MurMur3 has had at least some use, and it has a fast x64_128 method that at least gives us 128 bit instead of just 32.
>> Don't know if you feel it's worth spending time implementing it in FFmpeg though, I probably won't have time :-(
>> http://smhasher.googlecode.com/svn/trunk/MurmurHash3.cpp
> 
> 5th line of this says:
> // Note - The x86 and x64 versions do _not_ produce the same results, as the
> 
> any other suggestions ?

Why is that a problem? The hash optimized for 32 bit and only supporting 32 or 64 bit outputs has a different result than the one that gives up to 128 bit output and runs best on 64 bit.
Using the 64 bit version would probably mean a performance hit on a 32 bit system, but even there it should still be a lot faster than MD5 and I presume you mostly run 64 bit.