[FFmpeg-devel] [PATCH] improve DV probe score

[Reimar Döffinger]

On Mon, Sep 14, 2009 at 03:06:06PM +0200, Michael Niedermayer wrote:
> On Mon, Sep 14, 2009 at 09:59:31AM +0200, Reimar D?ffinger wrote:
> > Well, since that is the case for many formats I'd expected it is good
> > enough to even it out.
> 
> any formats i maintain?

Nothing I know for sure, it is just the impression, I haven't checked in
detail.
But I think the MP3 detection does not take buffer length into account
an in principle would detect anything as MP3 is the buffer is long
enough.
The MPEG-PS one too, for exactly the case where it caused issues (see my
other patch).

> if so, tell me and i see what i can do, but IMO foremost the
> ones that actually fail in practice (like dv here) should be fixed.

Well, actually I think the MPEG stuff is and always was (and may
necessarily "has" to be) the biggest offenders.

> > I don't like making such extensive changes since we have few DV samples,
> > but what about this change?
> 
> i think the code should check for invalid things that cannot occur in valid
> dv and then compare the valid vs. invalid counts to decide on the score

The problem with DV is that there is very, very little that has to be in
a certain way in reality (of course, according to the spec that's a
different thing).

> You argue that we dont have enough dv files, sorry to flame but
> your code cant even distinguish them from /dev/random nor this file here
> at hand

I would need the samples to know that real-world DV files will still be
recognized as such. I know that it sucks as it detects things as DV that
aren't but I don't know how far you can go before it will stop playing
real-world files (for DV aka. "files that barely resemble the spec").
We are already far stricter than any other DV player I know...

> also i cant see how the issue of too few samples could ever be solved
> except by changing the probe code to check more and seeing if that
> works in the wild or if we really have files with 99% of the startcodes
> missing all profile and framerate values changing mid frame and all the
> packs being at wrong positions (something our decoder would btw fail at)

I admit that once you have several frames in the buffer this could be
done more reliably. I would like to keep the detection at least barely
workable when a user wants or can provide only a rather small probe
buffer. Also I admit it is simply quite a bit of effort to check frame
sizes for validity etc.

> that said, while i disagree with how you solve this, i
> do not object to the patch, it is an improvment to what we have now
> i just think it can be done quite a bit more solidly, even a check
> of the number of startcodes per byte && the matches being > 3 would
> be better than what its now, also as you return a score >0 for a single
> match, am i missing something or shouldnt a single valid frame have at
> least 10 matches? it seems requireing 3 before a non 0 score is returned
> seems quite reasonable to me

How did you come up with the number 10? These checks are exactly like
that used to find the start of the frame, so there should be at most 2
matches per frame, and we have a sample with only one match per frame
IIRC (pond.dv, which when I tested has about 1 match per 100 kB, and I
don't like requiring > 300 kB of probe buffer before recognizing it at
all when it is not necessary - maybe we could/should reserve some lower
range of probe scores for "this is just a guess, retry with a larger
buffer unless you absolutely can't avoid it"?).