[FFmpeg-devel] [PATCH] Check rc_buffer_size value using integer comparison

[Mans Rullgard]

For certain input values, the floating-point value calculated
for bit_rate*time_base may, due to rounding, become minutely
larger than rc_buffer_size, thus causing the configuration to
be rejected in error.  Converting to int before the comparison
truncates any such overshoot.

This fixes the mxf regression test with gcc 4.5 on x86_32.
---
 libavcodec/mpegvideo_enc.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/libavcodec/mpegvideo_enc.c b/libavcodec/mpegvideo_enc.c
index 9f8682c..f55883b 100644
--- a/libavcodec/mpegvideo_enc.c
+++ b/libavcodec/mpegvideo_enc.c
@@ -351,7 +351,7 @@ av_cold int MPV_encode_init(AVCodecContext *avctx)
         av_log(avctx, AV_LOG_INFO, "impossible bitrate constraints, this will fail\n");
     }
 
-    if(avctx->rc_buffer_size && avctx->bit_rate*av_q2d(avctx->time_base) > avctx->rc_buffer_size){
+    if(avctx->rc_buffer_size && (int)(avctx->bit_rate*av_q2d(avctx->time_base)) > avctx->rc_buffer_size){
         av_log(avctx, AV_LOG_ERROR, "VBV buffer too small for bitrate\n");
         return -1;
     }
-- 
1.7.2.2