[FFmpeg-devel] [RFC v2 3/3] daaladec: Implement a native Daala decoder

[Ronald S. Bultje]

Hi,

On Tue, Dec 29, 2015 at 6:54 PM, Andreas Cadhalpun <
andreas.cadhalpun at googlemail.com> wrote:

> Hi,
>
> On 29.12.2015 19:46, Ronald S. Bultje wrote:
> > On Tue, Dec 29, 2015 at 11:49 AM, Andreas Cadhalpun <
> > andreas.cadhalpun at googlemail.com> wrote:
> >> Do you have a sample causing overflows in the vp9 decoder?
> >
> >
> > Nope, but I'm going to assume they're not hard to create, just use a few
> > high same-sign quantized coefficients (e.g. SHORT_MAX) and a high
> quantizer
> > (qidx=255, dq_ac=1828). Both dequant into int16_t (1828*SHORT_MAX doesn't
> > fit in 16 bits) as well as the idct results themselves (because you're
> > adding and subtracting from an already near-edge value which should fit
> in
> > 16bits) will overflow.
>
> I guess hard depends on who you ask. I've fuzzed the vp9 decoder and
> haven't yet come across a sample that triggers a signed integer overflow
> there.
> In the case of this daala decoder, 3/4 of the fuzzed samples do...
>
> >> (Overflows in dsp code are typically not a security concern.)
> >>
> >> Well, the overflows in the imdct calculation of the aac_fixed decoder
> >> ultimately
> >> caused crashes.
> >
> >
> > I would prefer if for video codecs, unless specifically proven otherwise
> > for that codec with specific content, we assume by default that overflows
> > in performance-sensitive dsp code (specifically idct) are OK. ubsan may
> not
> > like us on fuzzed content, but so be it.
>
> I prefer that input for such performance-sensitive dsp code gets sanitized,
> so that overflows are (nearly) impossible.


I don't think it makes sense to make the decoder 5% slower just so that
ubsan is happy. We don't do that for other video decoders either. And yes,
the coefficient decoding loop is typically a major hotspot in video codecs.

Ronald