[FFmpeg-devel] init_vlc_spare failing in huffyuv decoder, how to fix?

[Reimar Döffinger]

On Sat, Jul 04, 2009 at 11:59:27PM +0000, Loren Merritt wrote:
> On Sat, 4 Jul 2009, Reimar D?ffinger wrote:
> 
> > when decoding the smclockhuffyuv.avi.1.205 file from issue 1240 there is
> > a crash because (once more) the VLC tables 3,4 and 5 are NULL.
> > The reason is that init_vlc_sparse in generate_joint_tables fails.
> > Since that function "returns" void, does that mean that the bug is
> > elsewhere and it should never fail or are there just a lot of checks for
> > all those init_vlc* functions missing?
> 
> init_vlc_sparse can fail, but generate_joint_tables shouldn't give it 
> inputs that would fail, regardless of the stream contents. What actually 
> happened is that memory corruption changed the huffman tables after they 
> were validity-checked.

I suspected that might be the reason, which is why I asked.

> fixed.

Thanks.
If you or someone else by chance has the time, maybe you could confirm
if vp3.c:read_huffman_tree lacks any checks on the data and if it is ok
to just check the init_vlc_table return value to detect issues.
A sample file is ogv/smclock.ogv.1.101.ogv from issue 1240