[FFmpeg-devel] [PATCH 2/7] avcodec/flashsvenc: Account for header in packet size

[James Almer]

On 11/18/2022 11:59 PM, Andreas Rheinhardt wrote:
> Fixes ticket #10053.
> 
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at outlook.com>
> ---
>   libavcodec/flashsvenc.c | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/libavcodec/flashsvenc.c b/libavcodec/flashsvenc.c
> index 35793400fa..9d065bb92d 100644
> --- a/libavcodec/flashsvenc.c
> +++ b/libavcodec/flashsvenc.c
> @@ -229,7 +229,8 @@ static int flashsv_encode_frame(AVCodecContext *avctx, AVPacket *pkt,
>           I_frame = 1;
>       }
>   
> -    if ((res = ff_alloc_packet(avctx, pkt, s->image_width * s->image_height * 3)) < 0)
> +    res = ff_alloc_packet(avctx, pkt, 4U + s->image_width * s->image_height * 3);

For a 1x1 image (like the one from the ticket) this results in 7 bytes + 
padding being allocated...

> +    if (res < 0)
>           return res;
>   
>       pkt->size = encode_bitstream(s, p, pkt->data, pkt->size, opt_w * 16, opt_h * 16,

...yet encode_bitstream() will return 17, meaning it wrote into the 
padding bytes, which just happens to work because said padding was big 
enough. The smallest dimension that results in this being equal or 
smaller than the allocated size is 3x3.