[FFmpeg-cvslog] vmdav: Try to fix unpack_rle()

[Michael Niedermayer]

ffmpeg | branch: release/1.0 | Michael Niedermayer <michaelni at gmx.at> | Wed May  1 23:46:38 2013 +0200| [d814b9b51ddd3764e809c1d0f82b770e0bc085fd] | committer: Michael Niedermayer

vmdav: Try to fix unpack_rle()

This fixes out of array accesses
The code prior to this commit could not have worked, thus obviously
was untested. I was also not able to find a valid sample that uses this
code.
This fix is thus only based on the description of the format

If someone has a sample that uses unpack_rle(), please mail me.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni at gmx.at>
(cherry picked from commit c1f2c4c3b49277d65b71ccdd3b6b2878f1b593eb)

Conflicts:

	libavcodec/vmdav.c

Signed-off-by: Michael Niedermayer <michaelni at gmx.at>
(cherry picked from commit 0baa0a5a02e16ef097ed9f72bc8a7d7b585c7652)

Signed-off-by: Michael Niedermayer <michaelni at gmx.at>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=d814b9b51ddd3764e809c1d0f82b770e0bc085fd
---

 libavcodec/vmdav.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libavcodec/vmdav.c b/libavcodec/vmdav.c
index 3834416..b838781 100644
--- a/libavcodec/vmdav.c
+++ b/libavcodec/vmdav.c
@@ -160,7 +160,7 @@ static int rle_unpack(const unsigned char *src, int src_len, int src_count,
     const unsigned char *ps;
     const unsigned char *ps_end;
     unsigned char *pd;
-    int i, l;
+    int i, j, l;
     unsigned char *dest_end = dest + dest_len;
 
     ps = src;
@@ -186,9 +186,9 @@ static int rle_unpack(const unsigned char *src, int src_len, int src_count,
             ps += l;
             pd += l;
         } else {
-            if (dest_end - pd < i || ps_end - ps < 2)
+            if (dest_end - pd < 2*l || ps_end - ps < 2)
                 return ps - src;
-            for (i = 0; i < l; i++) {
+            for (j = 0; j < l; j++) {
                 *pd++ = ps[0];
                 *pd++ = ps[1];
             }