[FFmpeg-devel] [PATCH] dashenc: check pts to prevent division by zero error

[Alfred E. Heggestad]

this usecase will cause a division by zero trap:

1. dashenc has received one frame
2. os->max_pts and os->start_pts have same value
3. delta between max_pts and start_pts is 0
4. av_rescale_q(0, x, y) returns 0
5. this value is used as denominator in division
6. Bang! -> segfault

this fix checks that max_pts > start_pts.
the fix has been tested and works.

please review and suggest better fixes.

Signed-off-by: Alfred E. Heggestad <alfred.heggestad at gmail.com>
---
  libavformat/dashenc.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c
index f555f208a7..3b651b9514 100644
--- a/libavformat/dashenc.c
+++ b/libavformat/dashenc.c
@@ -1883,7 +1883,7 @@ static int dash_flush(AVFormatContext *s, int 
final, int stream)
 
st->time_base,
 
AV_TIME_BASE_Q));

-        if (!os->muxer_overhead)
+        if (!os->muxer_overhead && os->max_pts > os->start_pts)
              os->muxer_overhead = ((int64_t) (range_length - 
os->total_pkt_size) *
                                    8 * AV_TIME_BASE) /
                                   av_rescale_q(os->max_pts - os->start_pts,
-- 
2.20.1 (Apple Git-117)