[FFmpeg-devel] [PATCH] Optimization: support for libx264's mb_info

[Stefano Sabatini]

On date Friday 2023-05-19 10:19:03 +0000, Carotti, Elias wrote:
> 
> Hi again,
> I am sending this patch again (I had missed a check for NULL), could
> somebody please have a look at it? 
> 
> It is mainly an optimization when the encoder knows in advance that
> only portions of the whole frame changed and which areas actually did.
> 
> The patch allows a user to pass down information to libx264 information
> about which parts of a frame changed (with respect to the preceding
> one) to be exploited as a hint for P_SKIP-ing macroblocks which didn't
> change. 
> This information is encoded in the mb_info field of the x264_param_t
> much like the quant_offsets which are already used for the
> AV_FRAME_DATA_REGIONS_OF_INTEREST side information.

Please send a git format-patch (complete with commit log information)
to simplify integration.

[...]
> diff --git a/libavutil/mb_info.h b/libavutil/mb_info.h
> new file mode 100644
> index 0000000000..918cf167aa
> --- /dev/null
> +++ b/libavutil/mb_info.h
> @@ -0,0 +1,46 @@
[...]
> +#ifndef AVUTIL_MB_INFO_H
> +#define AVUTIL_MB_INFO_H
> +
> +#include <stddef.h>
> +#include <stdint.h>
> +#include "libavutil/avassert.h"
> +#include "libavutil/frame.h"
> +

> +typedef struct _AVMBInfoRect {

nit: strip the _ before the struct name

> +    uint32_t x, y;
> +    uint32_t width, height;
> +} AVMBInfoRect;
> +
> +/**
> + * Allocate memory for a vector of AVMBInfoRect in the given AVFrame
> + * {@code frame} as AVFrameSideData of type AV_FRAME_DATA_MB_INFO.

> + * The side data contains a list of rectangles for the portions of the frame
> + * which changed from the last encoded one. The rest will be hinted to be
> + * P_SKIP-ped.  Portions of the rects which are not on macroblock boundaries
> + * are not handled as P_SKIPS.
> + */

> +AVMBInfoRect *av_mb_info_create_side_data(AVFrame *frame,
> +                                          AVMBInfoRect *rects,
> +                                          size_t num_rects);

Probably this can be generalized with a flag defining the hinting type
(you might want either to specify the constant or non-constant rects),
in fact this data is pretty macro-block agnostic.

What about AVVideoHintInfo containing a hint (CONSTANT, VARIANT) and a
list of rects?