[FFmpeg-devel] [PATCH] avutil/pixdesc: fix incorrect strlen arithmetic

[Ganesh Ajjanagadde]

strlen returns a size_t, which is unsigned. If it is less than 2 for
some pixel format. wrap-around will happen and a bad pointer dereference
will take place.

Yes, this is at the moment theoretical, but nonetheless dangerous in my
view and the fix is very simple.

-------------------------------------------------------------------------------
Inspired by a patch from Andreas Cadhalpun, I am running an audit of the
FFmpeg codebase for fishy usage of the string handling functions.

Signed-off-by: Ganesh Ajjanagadde <gajjanagadde at gmail.com>
---
 libavutil/pixdesc.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c
index 72d0470..4e02c14 100644
--- a/libavutil/pixdesc.c
+++ b/libavutil/pixdesc.c
@@ -2232,12 +2232,13 @@ enum AVPixelFormat av_pix_fmt_swap_endianness(enum AVPixelFormat pix_fmt)
 {
     const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(pix_fmt);
     char name[16];
-    int i;
+    int i = 0;
 
     if (!desc || strlen(desc->name) < 2)
         return AV_PIX_FMT_NONE;
     av_strlcpy(name, desc->name, sizeof(name));
-    i = strlen(name) - 2;
+    if (strlen(name) >= 2)
+        i = strlen(name) - 2;
     if (strcmp(name + i, "be") && strcmp(name + i, "le"))
         return AV_PIX_FMT_NONE;
 
-- 
2.6.2