[FFmpeg-devel] [PATCH 2/5] fate: avoid framemd5, use framecrc its faster

[Michael Niedermayer]

On Wed, May 08, 2013 at 08:18:49PM +0200, Reimar Döffinger wrote:
> On Wed, May 08, 2013 at 07:05:08PM +0200, Michael Niedermayer wrote:
> > On Wed, May 08, 2013 at 06:24:25PM +0200, Reimar Döffinger wrote:
> > > Michael Niedermayer <michaelni at gmx.at> wrote:
> > > 
> > > >Signed-off-by: Michael Niedermayer <michaelni at gmx.at>
> > > 
> > > I disagree with this, CRC can easily miss structural changes,
> > 
> > Can you give an example of a structural change that has occured over
> > the lifetime of ffmpeg in one of its output files that would have
> > failed to be detected with CRC (or rather adler32)?
> 
> Certainly no actual examples, even if they existed finding
> one would be more effort that it is worth.
> But for CRC a major issue would be that any changes in parts protected
> by a CRC with the same polynomial would be undetectable.

> I have never researched adler32 so I can't say whether it might
> have similar issues.

adler32 is not affected by that, its affected by other things though
also ffmpeg uses adler32 under the name crc at several places


> 
> > The question is not if you can construct a pattern that gives a
> > crc equal to some other because noone sits there and tries to generate
> > such collisions, the checks are there to detect unintended bugs.
> 
> The real question is if there are structures used in multimedia
> formats or content that have a good chance of having collisions.

the real question is which checksum safes us more time.
md5 is better but crc & adler32 faster
Ive never seen crc or adler fail on real world data. It sure does
happen, for example you should find a colission in 100000 random files
but thats not the question.
The question is if debuging these bugs would cost more time than we
safe by the change.
If i safe 10minutes a day, that sure is alot more than i will ever
spend on debuging crc & adler32 colission related bugs.
and i can spend these 10minutes on debuging other bugs thus ...



> For CRC my answer would be "at the very least use an unusual
> polynomial and preferably 64 bit", but I can't answer that for
> adler32, however this paper:
> http://www.zlib.net/maxino06_fletcher-adler.pdf seems to claim
> it is pretty bad even on random data while also being slower than
> Fletcher-32.

> It also suggests that they may fail to detect e.g. padding changing
> from 0x00 to 0xFF. This might be relevant to the valgrind memory

do you have a example of such colission with changed padding ?


[...]

> An interesting question (though to be honest unless you want to
> I think I'd have you rather have you work on other stuff) would
> be if there is something more hashlike than adler32 with similar speed,
> RC4 maybe, I think it can be used for hashing?
> Anyway I leave it up to you, it's mostly a gut feeling from my side so
> far and not really worth wasting much of your time discussing it.

If you know of a better & fast method that has been subjected to
some serious use & testing then iam surely interrested.
Iam not interrested in spending time on researching or reinventing
one though. as far as iam concerned crc & adler are fine until a
problem is found, its also quite easy to switch back to md5 in that
case

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The bravest are surely those who have the clearest vision
of what is before them, glory and danger alike, and yet
notwithstanding go out to meet it. -- Thucydides
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20130509/6733983f/attachment.asc>