[FFmpeg-devel] [PATCH]: avoid sending empty probe data to probing functions

[Paul B Mahol]

On 7/4/12, Thomas Hutschenreuther <thutschenreuther at mufin.com> wrote:
> I have seen some rare occurrences of crashes in aacdec.c in the function
> adts_aac_probe.
> The crash occurs in line 45:
>
> uint32_t header = AV_RB16(buf2);
>
> The buf2 directly comes from the AVProbeData given to this function.
> Inspection showed that all fields of that AVProbeData instance were set
> to zero.
> Also in this function no checks are performed on the existence/validity
> of the probe data.
>
> As I did not want to check all probing functions for checks on input
> data, I looked for a possibility to prevent this at a higher level.
>
> The point where the probe data is assembled in my case is probe_codec()
> in libavformat/utils.c.
>
> This function is called from ff_read_packet().
> There, if
>
> ret= s->iformat->read_packet(s, pkt);
>
> in line 738 fails and there are already packets in pktl, probe_codec may
> be called with NULL as last argument.
>
> This may also be the first packet given to this function for a certain
> stream.
> If this happens, then the AVProbeData associated with this stream will
> be empty and will be forwarded to set_codec_from_probe_data().
>
> To avoid this, I suggest the following patch.
>
>
> diff --git a/libavformat/utils.c b/libavformat/utils.c
> index 812beeb..8eec8be 100644
> --- a/libavformat/utils.c
> +++ b/libavformat/utils.c
> @@ -694,7 +694,11 @@ static void probe_codec(AVFormatContext *s,
> AVStream *st, const AVPacket *pkt)
>                   || st->probe_packets<=0;
>
>           if(end || av_log2(pd->buf_size) != av_log2(pd->buf_size -
> pkt->size)){
> -            int score= set_codec_from_probe_data(s, st, pd);
> +            int score = 0;
> +            if( pd->buf_size > 0 )
> +            {
> +                score = set_codec_from_probe_data(s, st, pd);
> +            }
>               if(    (st->codec->codec_id != CODEC_ID_NONE && score >
> AVPROBE_SCORE_MAX/4)
>                   || end){
>                   pd->buf_size=0;
>

This is just bad workaround and does not fix anything.